Benchmarking Clinical Safety and Diagnostic Accuracy of Large Language Models in Otorhinolaryngology Emergencies: A Cross-Lingual Comparison of ChatGPT 5.2 and Gemini 3 Pro

BackgroundWhile the clinical potential of Large Language Models (LLMs) is increasingly recognised, their safety in morphologically rich languages remains debated. This exploratory study comparatively examines the ability of two recent artificial intelligence models, ChatGPT 5.2 and Gemini 3 Pro, to evaluate simulated otorhinolaryngology (ENT) cases across urgent, priority, and elective triage categories in Turkish and English.MethodsThe study was designed around 30 standardised clinical scenarios covering frequent ENT pathologies. Gold standard diagnosis and disease management decisions were determined by expert consensus. Model performance was evaluated in terms of three-category triage accuracy, clinical safety scores, and emergency-detection performance, using expert consensus decisions as the reference standard. Cohen's Kappa, Wilcoxon Signed Ranks, and McNemar tests were applied to measure inter-model and inter-language agreement.ResultsAll 120 model-generated primary diagnoses were confirmed as correct by the two ENT specialists. Correct triage classifications were obtained in 55/60 (91.7%) responses for ChatGPT 5.2 and 58/60 (96.7%) responses for Gemini 3 Pro. Agreement with expert-defined triage decisions was almost perfect (κ = 0.813-0.938). No statistically significant differences in the evaluated paired comparisons of triage accuracy or clinical safety scores were detected (all p > 0.05). Gemini 3 Pro correctly identified all 18 emergency scenarios in both language conditions.ConclusionsBoth models demonstrated high observed diagnostic and triage accuracy in a limited set of simulated ENT scenarios spanning different acuity levels. Although no statistically significant differences were detected in the evaluated comparisons, these findings do not establish equivalence between models or languages. These encouraging findings, particularly the high performance observed in Turkish, support further investigation of large language models as potential adjunctive clinical decision-support tools, while larger and more diverse clinical studies remain necessary before routine clinical implementation.