To systematically evaluate the diagnostic, classification, and treatment decision-making performance of large language models (LLMs) in dry eye disease (DED) and compare their performance with that of junior ophthalmologists to assess their feasibility as clinical support tools. One hundred standardized DED cases from Fuzhou University Affiliated Provincial Hospital (Aug 2023–Aug 2025) were analyzed. Four LLMs (ChatGPT-4o, DeepSeek-V3, Gemini-2.5-Pro, and ERNIE Bot-4.5-turbo) underwent an initial 20-item test with a 95% accuracy threshold. Only models meeting this criterion advanced to case-based evaluation. Diagnostic and therapeutic outputs were rated by senior ophthalmologists using the Global Quality Score (GQS) and a clinical safety score. Meanwhile, the response times of both the LLMs and the junior ophthalmologists were recorded to evaluate their respective efficiency. ChatGPT-4o, DeepSeek-V3, and Gemini-2.5-Pro met the screening threshold. In determining treatment necessity, the accuracies of the LLMs (96–99%) were comparable to those of junior ophthalmologists (97%). For DED classification, DeepSeek-V3 achieved the highest accuracy (92%) and agreement with experts (kappa = 0.80), outperforming ChatGPT-4o and junior ophthalmologists (71%, kappa ≈ 0.53; p < 0.01). Gemini-2.5-Pro showed strong performance (accuracy 89%), the highest GQS (4.87 ± 0.37), and the best safety rating, with 89% of its outputs rated as “good”. The decision efficiency of the LLMs was significantly higher (p ≤ 0.001), with response times (16–36 s) much shorter than those of junior ophthalmologists (315 ± 57 s). Gemini-2.5-Pro and DeepSeek-V3 demonstrated high accuracy, safety, and efficiency in case-based DED management, showing strong potential as auxiliary tools to enhance clinical decision-making and support less experienced clinicians. Not applicable. This study is a retrospective observational study and does not involve any intervention requiring trial registration.
Cui et al. (Tue,) studied this question.