Robustness of a neural network approved for skin cancer diagnosis against variations in sequential images

Summary Background : Convolutional neural networks (CNN) for skin cancer classification have shown results comparable to dermatologists but are vulnerable to minor image transformations. We investigated the robustness of a MDR class‐IIa certified CNN when classifying sequential images of identical lesions. Patients and Methods : We acquired 2,744 dermoscopic images of 385 skin lesions (80.8 % benign, 19.2 % malignant) and applied in‐vivo zoom, rotation (90‐degree increments), and simple repetitions of image recordings. Sequential images of identical lesions were classified by a binary CNN (Moleanalyzer‐Pro, FotoFinder Systems, Germany) and the variability of scores was investigated using intraclass correlation coefficient (ICC), mean absolute change of scores ( mac ), and probability of change of predicted class (). Results : In dermoscopic baseline images (n = 385) the CNN showed a sensitivity, specificity, and area under the receiver operating characteristic (AUROC) (95 % CI) of 91.9 % (83.4 %–96.2 %), 87.8 % (83.7 %–91.0 %) and 0.947 (0.921–0.972), respectively. The ICC across images of identical lesions was 0.872 (0.862–0.883), indicating excellent reliability. Overall mac of scores was 0.102 (0.090–0.115) and was 7.5 % (5.8 %–9.2 %). Conclusions : The tested CNN demonstrated a profound robustness against image variations as might be introduced during sequential digital dermoscopy. Clinically relevant class changes occurred in one of 13 images.