Dual stream spatiotemporal feature learning framework for neurological disease severity level classification using fMRI data

Neurological disorders such as Alzheimer’s disease, epilepsy, and Parkinson’s disease are often characterized by subtle yet progressive alterations in brain activity that can be effectively studied through functional magnetic resonance imaging (fMRI). Accurate classification of disease severity is critical for clinical decision-making and personalized treatment planning, but remains a challenging task due to the high dimensionality of fMRI data, dynamic brain connectivity patterns, and inter-subject variability. To address these challenges, this study introduces a novel multimodal framework that integrates both spatiotemporal neural activity and dynamic functional connectivity (dFC) representations. The first stream employs the AdaBelief Boosted Contrastive DenseConvo net (ABCD Net), derived from a 3D DenseNet backbone, enhanced with Temporal Contrastive Learning and ConvLSTM fusion for robust spatiotemporal feature embedding. In parallel, the second stream utilizes a Dynamic Slepian Gated Recurrent Transformer (DynSGRT) to construct windowed dFC graphs, which are further processed using the Hierarchically differentiated temporal-evolving graph neural network (HD-TEG Net) with graph pooling and temporal evolution modeling. The features from both streams are fused via a Cross-modal dual-head attentive concatenation strategy, followed by classification using a Chicken swarm optimized DEformer (Dynamic embedding transformer), which enhances generalization and avoids local minima through bio-inspired hierarchical search. Experiments were conducted on a balanced benchmark dataset comprising 800 subjects collected from NeuroVault and NITRC repositories, including Alzheimer’s Disease, Parkinson’s Disease, Epilepsy, and Normal (200 subjects each), using a subject-independent 70:30 train-test split and five-fold cross-validation, where all reported results were evaluated on completely unseen testing subjects to ensure unbiased performance assessment. Results demonstrate that the proposed model achieves superior performance with 99.36% accuracy, and an AUC-ROC of 0.991, significantly outperforming existing baseline and literature models. These findings highlight the potential of the proposed framework as an effective diagnostic tool for neurological disease severity classification, bridging spatiotemporal and connectivity-based perspectives of brain function.