A Dual-Branch Deep Learning Attention-Based Feature Fusion Approach for Alzheimer Severity Classification

Alzheimer's disease (AD) is known to be the most common form of dementia worldwide. Early and accurate classification of its severity stages from brain MRI scans remains challenging. Several deep learning solutions have been employed to solve this challenge. However, most of these existing methods are hampered by single-backbone architectures that capture limited feature representations. Hence, in this research, a dual-branch deep learning model that combines EfficientNetB4 and ResNet50V2 through an attention-based feature fusion mechanism (DBDAF) for multi-class AD classification from structural MRI images is presented. The model was trained and evaluated on the OASIS MRI dataset, which contains 86,437 images across four severity classes: non-demented, very mild dementia, mild dementia, and moderate dementia. A strict patient-level 70/15/15 data split was applied to ensure zero leakage between the training, validation, and test sets. The class-weighted loss was used to address the class imbalance in the dataset. On the held-out test set of 12,968 images, the model achieved a test accuracy of 98.99%, an AUC of 0.9998, and a weighted F1-score of 98.99%. Notably, both the MildDementia and ModerateDementia classes were classified with perfect recall, meaning no confirmed dementia case was missed. Grad-CAM visualizations showed that the model (DBDAF) attends to neurologically relevant brain regions, with activation patterns that reflect the known progression of the disease across severity stages.