Benchmark analysis reveals challenges in predicting regulatory variant effects in human microglia, indicating a decoupling of accessibility and activity.
Background. Sequence-to-function deep-learning models — DeepSEA and Basset in the first generation, and the long-context models Borzoi and AlphaGenome today — predict genomic assay tracks with high accuracy and are increasingly used to interpret the noncoding variants that underlie most disease-associated genetic risk. Their utility rests on a harder question than track prediction: can they predict the direction (sign) of a variant's regulatory effect in a defined cell context? Results. We assembled a chromosome-split benchmark of noncoding variants with laboratory-measured effect direction across nine cellular contexts (6,377 variant×context measurements), plus a native human-microglia caQTL map from 95 donors. On the fairest test — chromatin-accessibility direction in the native cell type — AlphaGenome (0.537), Borzoi (0.551), motif floor baselines, and in-distribution models were all indistinguishable from chance (majority-class 0.565). Same-donor caQTL vs eQTL sign concordance was 0.506 (95% CI 0.455–0.556; p = 0.87) — the layers are statistically independent. The Alzheimer's variant rs6733839 (BIN1) anchors the mechanism: the risk allele opens chromatin and raises BIN1 expression yet represses episomal enhancer activity; a Boltz-2 co-fold shows a tighter MEF2A–DNA interface at the risk allele (+12.5% contacts). Conclusions. Regulatory-variant effect-direction is limited not by model capacity but by an accessibility–activity decoupling that single-readout models cannot represent. We release the benchmark, the independence statistic, and all per-variant scores as a community resource.
No takes yet. Share an insight, caveat, or question.
Anuj Dev Singh (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: