Key result
Information leakage in EEG-based schizophrenia detection inflates classification accuracy by ~30% versus leakage-free pipelines.
Why the study?
Information leakage from erroneous evaluation practices in machine learning pipelines may cause overestimation of performance in EEG-based schizophrenia detection.
Does information leakage in machine learning pipelines overestimate schizophrenia detection accuracy from EEG signals?
Does information leakage in machine learning pipelines overestimate schizophrenia detection accuracy from EEG signals?
Effect estimate: inflated by up to ~30%
Information leakage in machine learning pipelines for EEG-based schizophrenia detection significantly inflates classification accuracy, highlighting the need for rigorous, leakage-free evaluation practices.
High reported accuracies for EEG-based schizophrenia detection likely reflect flawed evaluation; challenges validity of existing ML literature and requires subject-level validation standards.
Detecting schizophrenia (SZ) from electroencephalography (EEG) signals using machine- and deep learning models gained traction lately due to potential utility in early disease detection and differential diagnosis. Classification performance reports in the range of 95% accuracy and above are common; however, review of state-of-the-art literature indicates that ~65% of published works involve erroneous practices in the evaluation pipeline such as epoch- instead of subject-based data splitting or ranking and selecting features before data partitioning. The consequent information leakage can result in an overestimation of SZ detection performance. Here we explicitly test this on three open SZ-EEG datasets using gold standard classification approaches in leaky and leakage-free implementations. Results indicate that information leakage can inflate SZ classification accuracy by up to ~30%. Accordingly, best practices regarding EEG-based SZ detection must be established and promoted before this technology can be further developed into a clinical decision-making tool.
No takes yet. Share an insight, caveat, or question.
Rácz et al. (2026) studied Schizophrenia. Information leakage (e.g., epoch-based data splitting) vs. Leakage-free implementations was evaluated on Schizophrenia classification accuracy (inflated by up to ~30%). Information leakage in EEG-based schizophrenia detection, such as epoch-based data splitting, can inflate classification accuracy by up to ~30% compared to leakage-free implementations.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: