Answers are grounded in this paper and cite their sources.
Deep learning studies on neuroimaging data frequently report accuracies close to 100%, yet such results can arise from data leakage rather than from learned disease signal. This paper is a case study of that failure in the author’s own work. A 3D convolutional neural network (CNN) was trained to separate Alzheimer’s disease (AD), mild cognitive impairment (MCI), early MCI (EMCI) and cognitively normal (CN) participants using 191,428 individual 3D volumes extracted from functional MRI (fMRI) acquisitions in the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Each time frame was treated as an independent image, and the pooled frames were split at random, at the image level, into training and validation sets. The network reached 100% validation accuracy by epoch 14 and made no errors on any of the 191,428 volumes. We show that this outcome is explained by frame-level leakage: frames of the same participant are nearly identical, so a model can recognise individuals instead of disease stages, and a random split places almost every participant on both sides. We document the training signatures that exposed the problem, relate them to leakage patterns reported in the Alzheimer’s literature, quantify why image-level splitting of fMRI frames guarantees subject overlap, and propose a subject-level evaluation protocol. The corrected experiment is left to future work; the contribution of this paper is a transparent, reproducible account of how a plausible pipeline yields an invalid result, and how to detect it.
No takes yet. Share an insight, caveat, or question.
ARIFUL Islam (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: