Evaluation of a Computer-Aided Diagnosis System for Diabetic Retinopathy Screening on Public Data

PURPOSE: To evaluate the performance of a comprehensive computer-aided diagnosis (CAD) system for diabetic retinopathy (DR) screening, using a publicly available database of retinal images, and to compare its performance with that of human experts. METHODS: A previously developed, comprehensive DR CAD system was applied to 1200 digital color fundus photographs (nonmydriatic camera, single field) of 1200 eyes in the publicly available Messidor dataset (Methods to Evaluate Segmentation and Indexing Techniques in the Field of Retinal Ophthalmology (http://messidor.crihan.fr). The ability of the system to distinguish normal images from those with DR was determined by using receiver operator characteristic (ROC) analysis. Two experts also determined the presence of DR in each of the images. RESULTS: The system achieved an area under the ROC curve of 0.876 for successfully distinguishing normal images from those with DR with a sensitivity of 92.2% at a specificity of 50%. These compare favorably with the two experts, who achieved sensitivities of 94.5% and 91.2% at a specificity of 50%. CONCLUSIONS: This study shows, for the first time, the performance of a comprehensive DR screening system on an independent, publicly available dataset. The performance of the system on this dataset is comparable with that of human experts.