A realistic simulation-based benchmark of microbiome normalization in sample stratification and taxa-level analysis

Motivation: Normalization is a critical step in microbiome studies because sequencing depth and sparsity can strongly affect downstream analyses. In real datasets, however, the underlying biological signal is unknown, making it difficult to determine whether a normalization method preserves true group differences or introduces distortions. To address this problem in a way that remains relevant to real applications, we developed a simulation-based evaluation framework informed by real microbiome data. The framework generates realistic datasets with known ground truth and enables quantitative comparison of normalization methods at both the sample and taxa levels. Results: Method performance depended on taxonomic resolution and on whether sequencing depth was confounded with group structure. In our case study, model-based normalization-factor methods, particularly edgeR-TMM and, in some settings, DESeq2, gave the closest match to the simulated biological contrast, indicating better recovery of taxa-level differences while preserving sample-level separation. TSS and rarefaction were often the next-best performers. Shannon diversity analyses further showed that sequencing-depth differences alone could create false-positive group differences for several methods, whereas rarefaction remained closest to nominal Type I error control. These results also showed that visual or statistical sample separation alone was not sufficient to judge normalization performance, because apparent group differences did not always correspond to correct taxa-level recovery. Rather than identifying a universally best method, the proposed framework provides a coherent strategy for evaluating existing and new normalization approaches under realistic, data-dependent scenarios.