Nested co-expression network analysis identifies compact gene clusters in a black box

Motivation Digital analysis of biological systems requires methods capable of identifying both broad and nested gene modules reflecting complex biological processes. Existing transcriptomic methods often miss compact gene sets corresponding to subprocesses in specialized cell types, limiting insights into functional heterogeneity. Results We present Nested-WGCNA, a two-stage unsupervised network analysis algorithm designed to identify coarse-grained and fine-grained gene modules. Applied to bulk RNA-Seq data, Nested-WGCNA reveals stable modules reproducible across datasets. When validated against scRNA-Seq data, these modules correspond to both major and minor immune cell subtypes. Application to immunotherapy response datasets uncovers predictive and prognostic biomarkers, highlighting its utility in treatment stratification and biomarker discovery. Availability The NestedWGCNA source code and analysis pipeline are available on GitHub (https://github.com/ilyada/NestedWGCNA) and archived on Zenodo (https://doi.org/10.5281/zenodo.18959244). Supplementary information Supplementary data are available at Bioinformatics online.