NeuroCrossAttention fusion for multimodal explainable early diagnosis of neurodegenerative diseases

Alzheimer's disease (AD) and Parkinson disease (PD) are common neurodegenerative diseases with progressive cognitive or motor dysfunction, which have been recognised as a significant burden on public health. The phenotypically divergent manifestations of these disorders are multifactorial, incorporating risks from diverse sources (neuroimaging, genetics, clinical history, and EHRs), making it difficult to establish accurate and timely diagnoses. Existing deep learning-based methods are mainly unimodal or rely on simple fusion techniques, which do not exploit inter-modal dependencies and limit the generalizability of the datasets. They are hard to interpret and lack clinical applicability. To address these issues, we propose an innovative multimodal deep learning framework, i.e., NeuroCrossAttention Fusion (NCAF), that integrates images, genetics, clinical information, and EHR data via cross-modal attention. The neural network learns to automatically weigh the importance of different modalities in a task-dependent manner, taking into account their relationships. It can produce an effective fusion that surpasses classical early- or late-concatenation methods. For the model, we have an explainability module that outputs Grad-CAM for imaging and SHAP for tabular modalities, enabling clinicians to validate them. We conducted thorough experiments on four benchmark datasets (ADNI, PPMI, OASIS, and UK Biobank) in both intra-dataset and cross-cohort settings. The model yielded superior performance on each dataset and achieved the best (92.8\%, AUC = 0.964) among state-of-the-art deep learning baselines, including MultiModal-Transformer and Late Fusion Ensemble, on the ADNI dataset, with wide margins in accuracy. Ablation studies showed that the cross-attention and explainability modules both contributed more to performance than either did alone. The framework, with its promising interpretability and generalisation ability in clinical decision support (i.e., diagnosis of neurodegenerative diseases and monitoring disease progression), is believed to be a valuable tool for clinicians to diagnose patients. By addressing the limitations of previous methods, it offers a scalable and robust AI solution for real-world healthcare applications.