Using transcripts to refine image based cell segmentation with FastReseg

Abstract Spatial transcriptomics (ST) faces persistent challenges in cell segmentation accuracy, which can bias biological interpretations in a spatial-dependent way. FastReseg introduces a novel algorithm that refines inaccuracies in existing image-based segmentations using transcriptomic data, without radically redefining cell boundaries. By combining image-based information with 3D transcriptomic precision, FastReseg enhances segmentation accuracy. Its key innovation, a transcript scoring system based on log-likelihood ratios, facilitates the quick identification and correction of spatial doublets caused by cell proximity or overlap in 2D. FastReseg reduces circularity in boundary derivation, and addresses computational challenges with a modular workflow designed for large datasets. The algorithm’s modularity allows for seamless optimization and integration of advancements in segmentation technology. FastReseg provides a scalable, efficient solution to improve the quality and interpretability of ST data, ensuring compatibility with evolving segmentation methods and enabling more accurate biological insights.