Answers are grounded in this paper and cite their sources.
Background/Objectives: Machine-learning studies of autism spectrum disorder from resting-state functional magnetic resonance imaging routinely report near-perfect accuracy, yet the models rarely transfer to an unseen scanner. We audited a pipeline of the kind this literature builds, which reported an accuracy of 0.999, and traced that number to five data-leakage pathways. Methods: Under a nested leave-one-site-out protocol on 871 participants from the Autism Brain Imaging Data Exchange, in which the held-out site influenced nothing but model selection, the threshold, the combiner, or the site correction, we closed every pathway, priced each by reopening it, and validated on a locked external cohort. Results: With all five pathways closed, the graph attention network reached an area under the curve of 0.648 and did not outperform a support vector machine on the same connectivity (0.679; p = 0.009). Decision-level fusion added nothing detectable (0.682, p = 0.72). A network identified its training participants better than it detected autism (0.710 versus 0.610); both questionnaire benchmarks proved label-tautological, and performance held on the external cohort. Conclusions: Reported gains in this literature should be read as claims about evaluation protocols until shown otherwise on an untouched acquisition site; all predictions, models, and code are released.
No takes yet. Share an insight, caveat, or question.
Ubaida et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: