Distributional Coherence-Based Data Refinement for Early Detection in the Alzheimer’s Disease Spectrum Using SHAP-Guided Feature Selection

Alzheimer’s disease (AD) is a major type of cognitive impairment. Most existing studies on cognitive state classification focus on the direct application of various machine learning (ML) algorithms. These studies typically assume consistency in patient features across assessments, without explicitly accounting for the variability introduced by subjective evaluation. While a limited number of studies have attempted to address such variability, their applicability remains constrained. This study addresses the identified gap by proposing a distribution–coherent data refinement method. First, it incorporates SHAP-based feature selection, then excludes a subset of distributionally atypical records. These include records from patients with a single visit that contribute to increased intra-class variance and reduced inter-class separability, as well as records from patients with multiple visits that exhibit inconsistent longitudinal trajectories. The method operates on the training data, and the resulting refined training data is fed to ML algorithms. The experimental results show that detecting a small number of records (1.285%) leads to a minor enhancement in data quality. The Fisher score and Cohen’s f are increased by on average 0.016 and 0.011, while the means for inter-class and intra-class are improved by 0.012 and 0.004, respectively. Furthermore, the refined training data enables ML models to achieve a higher joint correctness rate of up to 7.1% compared with the original data. Additionally, ML models trained on the refined data exhibit improved classification performance, as reflected by an increase in the F1 score. Therefore, the proposed method provides a potential data refinement strategy for the longitudinal restricted cognitive impairment spectrum, specifically for those data describing normal cognition, early mild cognitive impairment, and late mild cognitive impairment.