Enhancing Accuracy in Classification Models for Skin Disease Diagnosis Based on Segformer and ConvNeXt Approach

This study introduces an innovative methodology to enhance the precision of skin disease diagnosis classification models by integrating segmentation results. Employing advanced machine learning techniques, our approach involves predicting lesion areas in skin images by combining SegFormer for skin lesion segmentation and backbone ConvNeXt for classifying skin images that consist of benign and malignant diseases. Based on training the SegFormer model for skin lesion segmentation, it achieved the IoU (intersection over union) ratio of 0.861 on the test set, outperforming the top 1 entry on the ISIC 2018 Leaderboards, which had an IoU of 0.802. Furthermore, our skin classification model uses image cropping to generate input images that emphasize damaged skin areas, eliminating redundant information. Leveraging the segmentation model’s results, we define the bounding box for the lesion area, obtain a new image within the bounding box by adding padding, and then compare this new data with the original data. The disease classification model, using ConvNeXt as its backbone, exhibited superior performance on the new dataset compared with the original dataset, achieving a higher accuracy of 1.61%, precision of 26.42%, and recall of 26.49%. This research paves the way for novel approaches to address disease diagnosis challenges in medical images, particularly in skin diseases. It can improve the performance of classification models when trained on image datasets that do not have synchronization during acquisition.