STRR-Net: A Swim-Transformer and Res2Net Architecture for Knee MR Images Multi-label Segmentation

Knee osteoarthritis (KOA) is a major cause of morbidity, especially in the older and female population. Automatic medical image segmentation is crucial for improving the diagnostic accuracy of KOA and optimizing treatment plans. Convolutional neural networks have evinced potential in enhancing the accuracy of KOA segmentation. However, CNNs face challenges in handling the complex variability in position, shape, and scale of segmentation targets. Additionally, existing CNN models struggle with accurate processing boundary information, which limits their effectiveness in clinical decision-making. To tackle these challenges, we introduce a deep-learning architecture known as STRR-Net, designed for the automated segmentation of knee joints in MRI scans. We also designed a novel parallel encoder structure to achieve complementary and enhanced feature extraction. In particular, we propose a skip-connection structure for multi-scale fusion, which allows us to simultaneously capture different levels of feature information. Our model attains an average Dice of 0.8176 for the knee dataset, with a score of 0.6653 for cartilage segmentation and 0.8320 for meniscus segmentation. Visualizing the feature maps allows for a detailed assessment of the knee joint’s anatomy, including cartilage thickness, thereby facilitating the timely identification of osteoarthritis.