Low-Dimensional Coexistence in Complex Microbial Ecosystems

The statistical physics view of microbial ecosystems often invokes random unstructured interactions, consistent with bacterial coexistence in very high-dimensional niche spaces. While this approach successfully reproduces broad community-level statistics, its central assumption of high-dimensional coexistence remains untested due to a lack of direct empirical estimates. Here we evaluate this assumption by inferring the effective niche dimensionality directly from abundance profile data using a joint species distribution model, a nonlinear low-rank matrix factorization approach. The inferred niche dimensionality, η D , tracks ecological expectations: It correlates significantly with the dimensionality of the metabolic environment across paired 16S-metabolomics datasets, supporting the abundance-only inference. Applying this framework across nearly 200 human gut microbiome studies, we find that the inferred dimensionality is consistently low (median η D ≈ 4 ), markedly lower than median η D ≈ 13 predicted by diversity-matched disordered consumer-resource models, directly contradicting the high-dimensional assumption underlying current theoretical models. Low dimensionality was associated with stronger interspecies competition and reduced biodiversity, consistent with the long-standing macroecological predictions. Consumer-resource simulations reproduced these empirical relationships only when extrinsic resource availability trade-offs were imposed, with species-intrinsic metabolic trade-offs contributing an additional reduction in the effective dimensionality. Together, our results challenge the high-dimensional assumption underlying current physics-based descriptions of microbiomes and identify low-dimensional coexistence as an essential feature of host associated microbiomes.