CellProfiler Analyst 3.0: accessible data exploration and machine learning for image analysis

1With the increasing adoption of high-throughput microscopy, scientists have been able to generate large datasets containing thousands of individual images. This necessitates automated computational analysis to efficiently derive biological insights from the raw data. Free software packages such as ImageJ and CellProfiler allow users to extract hundreds or thousands of numerical measurements from their image data, but it is ultimately on the user to determine which of these features are relevant to the biological problem being investigated. Spreadsheet programs are familiar but lack features and integration with source images that biologists often need. CellProfiler Analyst is a data exploration package for helping users to explore and extract information from large datasets, including (but not limited to) those produced by CellProfiler pipelines. The software includes several tools for users to visualize and filter their datasets, alongside tools for training machine learning classifier models within a convenient graphical user interface that is geared toward working with image data. While other tools such as Ilastik and Advanced Cell Classifier can provide a GUI for training classifiers with image-based data, these do not include data exploration and visualization tools like those in CellProfiler Analyst. These tools provide an intuitive interface for scientists to explore their data in forms such as histograms and scatter plots, though without the advanced statistical tools or extreme customizability of a pure programming language. Herein we present CellProfiler Analyst 3.0, which includes major performance improvements and new features that improve the utility of the software.

2We ported CellProfiler Analyst to the Python 3 programming language to ensure compatibility with future operating systems after the official Python 2 end-of-life in 2020. We also revised the program’s builds to package all Java dependencies within the main installer, which dramatically simplifies the installation process. In addition, we added a faster imageio-based image loader to supplement the existing bioformats-based implementation. While bioformats provides broader file format compatibility, imageio allows for common image formats to be loaded more efficiently, which greatly improves the time to generate object thumbnails. At present neither of these loaders supports accessing only partial sections of an image, which may be an area for further development to handle large files produced by whole slide imaging.

3CellProfiler Analyst 3.0 can export machine learning models that are compatible with CellProfiler 4.2+, allowing the resulting classifiers to be directly embedded into CellProfiler pipelines. This maintains and expands upon the interoperability between the two programs.

4We added a dimensionality reduction tool to help users visualize the variance within high-dimensional datasets (Fig. 1A). This is particularly important when creating classifiers that can identify rare images in a set; rare outliers are needed for training but may be difficult to find through random sampling. When the dataset contains thousands of measurements users can struggle to identify those which reveal these outliers. Dimensionality reduction allows the numerous measurements generated by CellProfiler to be condensed into a smaller subset of features which represent the overall variance in the dataset. This provides a more manageable series of features which the user can then explore.

5This tool supports multiple common reduction techniques including principal component analysis, Feature Agglomeration and t-Distributed Stochastic Neighbor Embedding. The resulting components can be visualized on a scatter plot, which can color individual objects by class if the classification was previously performed on the dataset. As with other tools, users can create gates to isolate specific populations for further analysis.

6We improved the gating functionality so that gates drawn in plotting tools are immediately made available within the Classifier tool, rather than having to be manually converted into filters. The dimensionality reduction tool contains a new lasso tool for selecting objects of interest through drawing a custom polygon. Right-click menu options allow the selected objects to be sent directly to an open Classifier window for use as training data. These improvements simplify the process of finding populations of interest and creating training sets for machine learning models in the Classifier.

7In the Classifier tool we added support for automatic rescaling of input data, based on the scikit-learn StandardScaler implementation. This improves the performance of certain classifier types (such as K-Neighbors) that are skewed by the absolute values of measurements. It also now supports tunable neural network classifiers based on the scikit-learn MLPClassifier class, which can be useful for performing complex non-linear classification tasks. New keyboard shortcuts for sorting objects into classification bins provide a more rapid means of generating training sets than dragging-and-dropping.

8We improved performance in several areas of the Classifier tool. We optimized loading of previously saved training sets by batching objects during database fetching, reducing the loading time for a sample dataset by over 10-fold (Fig. 1B). We also found that removing redundant database calls and unnecessary caching steps during classifier model training reduced processing time by 90% (Fig. 1C).

9Database handling improvements were also made in the functions for scoring the datasets after training. These refinements produced a modest improvement in the performance of the ‘score all’ (Fig. 1D) function. We further optimized the ‘score image’ function by revising the workflow for fetching object coordinates, which significantly reduced the time taken to display a scoring preview overlay (Fig. 1E). Usability improvements to the image viewer include a shortcut to resize the image to fit the window and a table display outlining counts of each class found in the image.

10The next major frontier for phenotype classification is to train deep learning models straight from raw pixels rather than requiring a separate feature extraction step. Enabling scoring of phenotypes that fall along a continuum rather than into discrete bins would also be useful, as we recently found in red blood cell aging. A remaining limitation in our software is that only MySQL and SQLite databases can be accessed, though migrating toward using packages such as sqlalchemy could expand on compatibility in the future. Ultimately, maintaining CellProfiler Analyst as an up-to-date resource for users to explore high-dimensional, image-based data without needing to code will help the biology community for years to come.