Multi-scale multi-attention network for diabetic retinopathy grading

Objective. Diabetic retinopathy (DR) grading plays an important role in clinical diagnosis. However, automatic grading of DR is challenging due to the presence of intra-class variation and small lesions. On the one hand, deep features learned by convolutional neural networks often lose valid information about these small lesions. On the other hand, the great variability of lesion features, including differences in type and quantity, can exhibit considerable divergence even among fundus images of the same grade. To address these issues, we propose a novel multi-scale multi-attention network (MMNet). Approach. Firstly, to focus on different lesion features of fundus images, we propose a lesion attention module, which aims to encode multiple different lesion attention feature maps by combining channel attention and spatial attention, thus extracting global feature information and preserving diverse lesion features. Secondly, we propose a multi-scale feature fusion module to learn more feature information for small lesion regions, which combines complementary relationships between different convolutional layers to capture more detailed feature information. Furthermore, we introduce a Cross-layer Consistency Constraint Loss to overcome semantic differences between multi-scale features. Main results. The proposed MMNet obtains a high accuracy of 86.4% and a high kappa score of 88.4% for multi-class DR grading tasks on the EyePACS dataset, while 98.6% AUC, 95.3% accuracy, 92.7% recall, 95.0% precision, and 93.3% F1-score for referral and non-referral classification on the Messidor-1 dataset. Extensive experiments on two challenging benchmarks demonstrate that our MMNet achieves significant improvements and outperforms other state-of-the-art DR grading methods. Significance. MMNet has improved the diagnostic efficiency and accuracy of diabetes retinopathy and promoted the application of computer-aided medical diagnosis in DR screening.