Automated classification of site-specific cutaneous photodamage using a convolutional neural network and 3D total body photography

Abstract Background Early detection of melanoma presents a major public health challenge. Growing evidence supports targeted surveillance of high-risk individuals identified using risk stratification. Skin photodamage is the primary environmental risk factor for melanoma, however, it is inconsistently captured and often relies on self-reporting or subjective observations, resulting in poor reproducibility. The increasing use of total body photography (TBP) in clinical skin examinations, combined with advances in artificial intelligence technology, presents new opportunities for automated skin assessment of UV damage. Objectives To develop a clinical photonumeric scale for photodamage assessment, use the scale to build a dataset of annotated image tiles, and train a convolutional neural network (CNN) to automate photodamage assessment from 3D-TBP. Methods Our photonumeric scale was validated for assessing photodamage and pigmentation from 3D-TBP by comparing inter-rater reproducibility between two dermatology research students and two laypeople. A total of 24,720 cutaneous image tiles from 56 individuals at high-risk and 51 at population-risk for melanoma were annotated. Annotated images were used to train a CNN with a multi-task learning (MTL) strategy that incorporated pigmentation as an auxiliary task to increase the performance for photodamage. The MTL-CNN was compared to a single-task CNN that considered photodamage in isolation. Results Laypeople achieved substantial-to-almost perfect agreement to dermatology research students using the photonumeric scale (κ=0.77-0.83). The MTL-CNN design improved performance compared to the single-task CNN with the area under the receiver operator curve (ROC-AUC) increasing from 0.91 to 0.96 (p0.01). Class-specific accuracy improved for mild (0.96 to 0.98, p=0.04), moderate (0.85 to 0.92, p0.01), and severe (0.97 to 0.99, p0.01) photodamage categories, and was maintained across each body site (range 0.86-0.92). Accuracy was reproduced in an external validation set with a ROC-AUCof 0.93, including class-specific accuracies of 0.97 for mild, 0.85 for moderate, and 0.97 for severe photodamage. An interface was developed to display CNN-labelled photodamage as heatmaps on 3D-TBP patient avatars for clinical interpretation. Conclusions Our CNN provides a novel tool to automatically and reproducibly report an individual’s photodamage phenotype from 3D-TBP. Incorporating this assessment into risk prediction models may inform targeted risk prediction facilitating surveillance recommendations.