Development of a universal imaging “phenome” using shape, appearance and motion (SAM) features and the SAM Phenotype Observation Tool (SPOT)

Cells are plastic, highly heterogeneous and change over time. High-content timelapse imaging promises to reveal dynamic cell behaviors, enabling more accurate identification of cell state and cell fate prediction for biological hypothesis generation and perturbation screens. To empower live-cell imaging based screening, we report the development of (1) a Shape, Appearance, Motion (SAM) “phenome”; a universal set of 2185 image-derived features that act as a image-“transcriptome” to comprehensively quantify an object’s instantaneous phenotype; (2) the SAM-Phenotype-Observation-Tool (SPOT), for image-“sequencing” analysis of phenomes. We validate the effectiveness of unbiased SAM-SPOT workflow on publicly available computer vision and 2D single cell imaging datasets. Importantly, we demonstrate that SAM-phenome outperforms features generated by deep learning AI models trained on >1 million fixed single cell and >5000 single cell video frames, respectively. SAM-phenome and SPOT deliver high-throughput, object-treatment-agnostic, comprehensive screening readouts of dynamics, promising to advance novel molecular target discovery and new medicine development.