Real world practice of Artificial intelligence Diagnostic System for Diabetic Retinopathy in Taiwan

Purpose: We evaluated the alterations of applying artificial intelligence (AI) diagnostic system for diabetic retinopathy (DR) screening in real-world practice. Methods: This retrospective study included 11,713 diabetic patients from the government-led Diabetes Shared Care Network. The AI system Verisee was integrated into the clinical workflow to identify referable diabetic retinopathy (RDR). Its performance was compared with ophthalmologist grading at the patient level using sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC). Subgroup analysis was performed by age and gender, with additional referral diseases identified by ophthalmologists. Results: Verisee achieved a sensitivity of 0.88, specificity of 0.86, accuracy of 0.86, PPV of 0.58, NPV of 0.97, and AUC of 0.87 in detecting RDR. Performance declined with increasing age, whereas sex distribution remained consistent across age groups. The AI system identified a higher proportion of RDR than ophthalmologists (27.45% vs. 18.15%). In addition to 1,818 patients with RDR, ophthalmologists identified other referral-warranted ocular conditions in 4.5% of cases. The AI system referred age-related macular degeneration (grades 2–4), whereas referral decisions for macular hole and macular edema (grades 1–2) varied; however, glaucoma (grades 0–1) identified by clinicians was not consistently referred. Conclusion: Verisee demonstrated high accuracy in detecting RDR but exhibited reduced performance in older patients. It had a higher referral rate than ophthalmologists yet missed certain conditions such as glaucoma. Despite effectiveness in DR screening, further refinement is required to support broader ophthalmic disease detection.