CellPatch: A Flexible and Efficient Framework for Single-Cell Foundation Model Empowered by Heuristic Gene Patching

Abstract The rapid advancement of foundation models has significantly enhanced the analysis of single-cell omics data, providing deeper insights into the complex cellular and genetic interactions across diverse tissues. However, existing foundation models often exhibit excessive complexity, which hinders their practical utility for end-users. Here, we present CellPatch, a lightweight foundation model that integrates cross-attention mechanism with patch tokenization to significantly reduce computational complexity during both pretraining and fine-tuning stages and preserve biological interpretability. Comprehensive evaluations on single-cell RNA sequencing datasets from multiple organs and tissue states demonstrate that CellPatch coupled with various decoders achieves state-of-the-art performance in direct prediction tasks. The modular design of CellPatch enables it as a versatile framework through either linear, adaptor or task-specific decoders, achieving high transfer learning performance across a wide range of biological tasks.