The U-shaped Transformer architecture for precise retinal vessel segmentation

Abstract Background: Retinal blood vessels serve as crucial biomarkers for many ophthalmic and cardiovascular diseases. Therefore, the development of automatic segmentation models for computer-aided diagnosis holds significant importance. However, existing CNN-based methods, such as U-Net, face challenges in capturing long-range dependencies due to limited receptive fields and inherent biases of convolutional operations. Recently, numerous Transformer-based techniques have been integrated into the U-Net architecture to overcome this limitation by efficiently capturing global feature correlations. However, the self-attention mechanism of Transformers requires substantial computational resources, and integrating Transformers with U-Net may result in a significant increase in model complexity. Many resource-constrained devices may not be capable of handling precise image segmentation tasks, and the substantial increase in computational complexity also brings the risk of overfitting. To address these challenges, we propose a model based on lightweight Transformers and residual networks (MobileViTv2-ResUNet) for accurately segmenting retinal blood vessels. Methods: Firstly, the algorithm preprocesses images using photometric distortion, simulating various lighting conditions and camera settings that may occur during retinal fundus image acquisition. This enhances the model's adaptability and robustness to real-world scenarios. Secondly, lightweight Transformer modules (MobileViTv2 blocks) are integrated into the encoding stage to replace convolutional modules for feature extraction. Next, Inverted Residuals are introduced in the encoder section to not only perform downsampling operations and reduce computational complexity but also enhance the network's representation and generalization capabilities. Subsequently, ASPP modules are inserted between the encoder and decoder to effectively fuse feature information from different scales. Lastly, our designed LeakyRes modules are incorporated into the decoding stage to prevent the occurrence of "neuron death" phenomena and improve the accuracy of retinal blood vessel segmentation. Results: Validated on the public datasets HRF、STARE and DRIVE, our MobileViTv2-ResUNet achieves segmentation accuracies of 77.20% and 79.67% for retinal blood vessels, with Intersection over Union (IoU) scores of 66.99% and 69.57%, and recall rates of 77.20% and 79.67%, respectively. Conclusion：Experimental results indicate that our method excels in improving existing vascular segmentation techniques, particularly in handling images with lesions, branches, crossings, and tiny vessels. It provides more accurate and detailed information on retinal vascular distribution. This advancement aids doctors in formulating personalized treatment plans for patients, thereby significantly enhancing treatment efficacy and patient experience.