Vision transformer and CNN-based skin lesion analysis: classification of monkeypox

Abstract Monkeypox is an important health problem. Rapid diagnosis of monkeypox skin lesions and emergency isolation when necessary is essential. Also, some skin lesions, such as melanoma, can be fatal and must be rapidly distinguished. However, in some cases, it is difficult to distinguish the lesions visually. Methods such as dermoscopy, high-resolution ultrasound imaging, etc. can be used for better observation. But these methods are often based on qualitative analysis, subjective and time-consuming. Therefore, in this study, a quantitative and objective classification tool has been developed to assist dermatologists and scientists. The proposed system classifies seven skin lesions, including monkeypox. A popular approach Vision Transformer and some popular deep learning convolutional networks have been trained with the transfer learning approach and all results have been compared. Then, the models that show the best accuracy score have been combined to make the final prediction using bagging-ensemble learning. The proposed ensemble-based system produced 81.91% Accuracy, 65.94% Jaccard, 87.16% Precision, 74.12% Recall, and 78.16% Fscore values. In terms of different criteria metrics, the system produced competitive or even better results than the literature.