Comparative Accuracy and Safety of Large Language Models in Glaucoma Knowledge Assessment

Précis: Across glaucoma MCQs, GPT-5.2, Gemini 3 Pro, and Claude Opus 4.5 were highly accurate, whereas DeepSeek-V3.2 underperformed; potentially harmful wrong answers were uncommon. Purpose: To compare the accuracy and potential patient-safety risk of four large language models (LLMs) on glaucoma multiple-choice questions (MCQs). Methods: DeepSeek-V3.2, GPT-5.2 (via ChatGPT), Gemini 3 Pro, and Claude Opus 4.5 were tested on two American Academy of Ophthalmology datasets: 129 text-only AAO ONE Network glaucoma self-assessment questions and 32 text-only Basic and Clinical Science Course (BCSC) glaucoma study questions. Each item was queried once per model using a standardized prompt in a fresh session. Accuracy was compared using Cochran’s Q and pairwise exact McNemar tests with Holm adjustment. Potentially harmful wrong answers (HW2) were coded using a prespecified rubric. Results: In AAO ONE, accuracy was 69.0% for DeepSeek-V3.2, 93.0% for GPT-5.2, 92.2% for Gemini 3 Pro, and 96.1% for Claude Opus 4.5 (Q=65.17, P <0.0001). DeepSeek-V3.2 was significantly less accurate than the other three models; differences among the other models were not significant. In BCSC, accuracy was 65.6%, 93.8%, 84.4%, and 87.5%, respectively (Q=18.00, P =0.0004); GPT-5.2 outperformed DeepSeek-V3.2 after correction. A post hoc item-level screen classified 38/161 questions (23.6%) as directly HW2-eligible; four HW2 events occurred and did not differ significantly across models. Conclusions: GPT-5.2, Gemini 3 Pro and Claude Opus 4.5 achieved high accuracy on text-only glaucoma MCQs, whereas DeepSeek-V3.2 underperformed. Harmful errors were uncommon but present, underscoring the need for clinician oversight.