A human-centred spectral enrichment framework for enhanced patient-focused skin cancer screening using deep learning

Early diagnosis of aggressive melanoma subtypes (acral lentiginous melanoma (ALM), melanoma in situ (MIS), nodular melanoma (NM) and superficial spreading melanoma (SSM)) is paramount for effective patient management and treatment. But, conventional RGB dermoscopic imaging has its limits due to the limited spectral information that is obtained with standard imaging systems. This study presents a computational spectral enrichment approach called Spectrum-Aided Vision Enhancer (SAVE), which can create simulated hyperspectral and narrow-band like images from conventional RGB images, thus enhancing image characteristics for further analysis by artificial intelligence (AI) based methods without the need for dedicated hyperspectral imaging hardware. A single-center dataset of 915 dermoscopic images from 4 melanoma subtypes was used to evaluate the performance of 5 hybrid deep-learning architectures: XGBoost_VGG16, SVM-DenseNet121, XGBoost_EfficientNetB0, SVM_InceptionNet-ResNetV2 and SGD_ResNet34. The classification results showed that using SAVE-based representations resulted in better classification accuracy than using the current RGB-based representations in various model architectures, with the best classification accuracy of 95% obtained by SVM_InceptionNet-ResNetV2 model. The results indicate that the computational spectral enrichment using RGB approach may offer extra discriminative information to aid the sub-type classification of melanoma, especially for the visually challenging lesions like MIS and SSM. In the current study, however, it is a preliminary evaluation from a single center, and the results should be validated by larger multi-center datasets, independent external systems, comparison with gold standard HSI systems and prospective clinical assessments to ensure this framework, called SAVE, is generalizable and clinically applicable.