On the illusion of scRNA-seq batch effect correction

Batch correction methods in single-cell RNA sequencing are essential for removing technical variation that can otherwise lead to misleading downstream analyses. The reliability of these methods is typically evaluated using unsupervised metrics. Here, we apply a Machine Learning (ML) technique called probing, formalized as the Batch Probing Score (BPS), to empirically demonstrate across six datasets that the most popular batch correctors fail to fully remove batch signal. In most cases, the batch of origin remains clearly identifiable after correction, even though standard evaluation metrics cannot detect it. We show that existing unsupervised metrics lack the sensitivity and specificity required to capture residual batch signal, whereas ML-based approaches can still detect it. This residual signal can similarly be picked up by downstream analysis tools, potentially leading to biased results. Because BPS is supervised, it directly quantifies batch signal strength by measuring how accurately the batch of origin can be predicted for each sample. It therefore provides an upper bound on the residual ML-actionable batch signal that could otherwise remain unnoticed. Our findings suggest that probing-based metrics should become a standard for assessing batch correction methods in single-cell RNA-seq and other areas of genomics. Significance statement Batch correction is used in virtually every single-cell RNA-sequencing study, yet its effectiveness is often assumed rather than rigorously tested. Removing unwanted technical signal is fundamentally ill-posed, and our results show that widely used correction methods frequently leave substantial batch information behind. Critically, standard evaluation approaches, including visual inspection with UMAP and commonly used quantitative metrics, often fail to detect this residual signal. This means downstream analyses may still exploit technical artifacts and produce biased biological conclusions while appearing properly corrected. By introducing the Batch Probing Score, we provide a more sensitive way to expose this hidden signal. Our study raises an urgent methodological concern for the entire single-cell community and challenges a foundational assumption underlying current analysis pipelines.