Cross-attention-based multimodal fusion for early diabetic retinopathy detection

Early detection of diabetic retinopathy (DR) is crucial for preventing irreversible vision loss; however, existing automated methods often rely on single-modality inputs, thereby limiting the complementary diagnostic value of multimodal imaging. This study presents a cross-attention-based multimodal fusion framework that investigates cross-attention-based representation learning using semantically aligned fundus photography and OCT datasets for the classification of early-stage diabetic retinopathy (E-DR). The proposed architecture utilises EfficientNetB0 and DenseNet121 as modality-specific encoders, followed by a mid-level cross-attention module that dynamically aligns and fuses vascular and structural features. To enable consistent multimodal training, we develop a harmonised label-mapping strategy and a balanced dataset construction pipeline. Extensive experiments across unimodal and fused configurations demonstrate that our fusion model significantly outperforms baseline models, achieving an overall accuracy of 96% and notable improvements in early-stage detection (e.g., F1-score of 83% for Stage 1), comparing to the previous fusion state-of-the-art works that rely on simple concatenation or late-stage integration, our proposed fusion model captures richer cross-modal feature interactions, resulting in improved classification performance on the evaluated datasets. This study provides an exploratory investigation of cross-attention-guided multimodal fusion for retinal image analysis for early DR diagnosis and other ophthalmic screening assessments.