Conflicting results between the analysis of skin lesions using a mobile-phone application and a dermatologist's clinical diagnosis: a pilot study

Dear Editor, The use of mobile‐phone health applications (apps) has become very popular in the general population. The SkinVision app (SVA, http://www.skinvision.com) is the first skin cancer app with a European CE certification, indicating that the product has met EU health, safety and environmental requirements. SkinVision lets the consumer ‘take photos of moles and skin conditions to analyse for skin cancer risk’. Artificial intelligence is used to interpret the modified image automatically. Based on this analysis, it evaluates lesions as low (green), medium (yellow) or high risk (red). The purpose of our pilot study was to investigate the degree of agreement between the ratings of the SVA and a dermatologist in skin lesions selected by patients. We included new patients referred by the general practitioner to our pigmented lesion clinic who met an automated triage system for pigmented lesions. The written, informed consent of all participants was obtained. The study was approved by the local medical ethical committee. The patients were asked to point out a maximum of two lesions that concerned them. The lesions were imaged by the researcher with a Samsung S4 mini smartphone with an 8·0‐megapixel autofocus camera (Samsung, Suwon, South Korea) and the photos were analysed by the SVA with an instant rating of green, yellow or red. Subsequently, the patients were seen by a dermatologist who specialized in skin cancer, who also gave a green, yellow or red rating and documented the diagnosis. The diagnosis was followed by histopathology if the dermatologist rated the lesion as red. Data were collected in Excel, checked for outliers and re‐evaluated for correctness. Descriptive statistics and measure of agreement (weighted kappa) were performed in SPSS 23 (IBM, Armonk, NY, U.S.A.). In total, 151 lesions were evaluated, 53% were from women with a mean age of 42·5 years (range 19–81). The interobserver agreement between the ratings of the app and the dermatologist was very low (weighted kappa = 0·073), with a 95% confidence interval of −0·55–0·205 and a P‐value of 0·121 (Table 1). This interobserver agreement is compatible with mere chance. Overall, 44% of the lesions with a red rating and 60% of those with a yellow rating analysed by the SVA were diagnosed as a common naevus by the dermatologist. Pathology tests were made in 9%, 14% and 19% of the lesions given a green, yellow and red rating by the app, respectively. One patient refused excision for a lesion that was diagnosed as a basal cell carcinoma (BCC) by the dermatologist. The app rated two of the three BCCs that were confirmed by histopathology as moderate risk and four of the seven histologically proven dysplastic naevi as low risk. The dermatologist rated all three BCCs as high risk, and one of the seven dysplastic naevi was rated as low risk. Results of ratings using the SkinVision app (SVA) vs. a dermatologist's diagnosis, showing poor interobserver agreement Percentage of SVA rating that got a green*/yellow**/red*** rating by the dermatologist. Results of ratings using the SkinVision app (SVA) vs. a dermatologist's diagnosis, showing poor interobserver agreement Percentage of SVA rating that got a green*/yellow**/red*** rating by the dermatologist. Maier et al. found a sensitivity of 73% and a specificity of 83% for melanoma using the SVA compared with histological results.1 We believe that the main reason for the difference between our negative and their positive results is the method of lesion selection. They selected melanocytic lesions that the dermatologist found so suspicious that they needed excision and most benign melanocytic lesions like banal naevi and benign nonmelanocytic pigmented lesions had already been excluded from further examination.1 As the SVA is intended to analyse ‘moles and skin conditions … for skin cancer risk’, the general population is likely to use the app for all lesions they find suspicious, as it is difficult for a lay person to distinguish between potential skin cancer and a banal naevus, seborrhoeic keratosis, fibroma, angioma and other benign pigmented lesions. Therefore, we believe that the effect seen in our study would be even stronger in the general population. The SVA missed two of three BCCs. This was possibly because the BCCs were only slightly pigmented. As patients will most probably use this app not only on strongly pigmented lesions, less‐pigmented types of skin cancer like amelanotic melanoma and BCC may be missed. Our results are in line with the study conducted by Wolf et al., who warned about the potential harm these apps could bring about.2 A recent review article about practical apps for diagnosing noninvasive melanoma found that many apps were not reliable.3 Our study has certain limitations. In cases of a benign clinical diagnosis, lesions were not excised for ethical reasons. Another limitation is the lack of melanoma cases due to the limited number of patients in the series. However, we expect that in real‐life situations the proportion of benign lesions is higher than in an academic hospital setting. The rating of the SVA is to be questioned. The high amount of false‐positive ratings could increase the use and cost of health care by worried members of the general public. False‐negative results may cause delays in diagnosing melanomas and other types of skin cancer. The authors wish to thank Ron Wolterbeek for his contribution to the statistical analyses. Funding sources: none. Conflicts of interest: none.