Could AI chatbots mislead orthodontic patients? An assessment from the perspectives of accuracy, risk, and ethics

The aim of this study is to comparatively evaluate the performance of three different Large Language Models (ChatGPT, Gemini, and Copilot), commonly used in orthodontic patient counseling, in terms of clinical accuracy, potential clinical risk, and ethical suitability. This prospective, cross-sectional, and scenario-based study utilized a total of 100 questions collected from orthodontic patients, categorized into five main groups (general knowledge, treatment process, emergencies, pediatric and adult orthodontics, special cases). The questions were administered to three different Large Language Models using a “zero-shot” approach between February and March 2026. The responses were evaluated by three experienced orthodontists using a 5-point Likert scale in a blinded evaluation process for accuracy, potential clinical risk, and ethical appropriateness. Data were analyzed using Repeated Measures ANOVA, and inter-observer agreement was calculated using the Intraclass Correlation Coefficient (ICC) ( p ≤ 0.05). When all categories and the total dataset were evaluated, no statistically significant differences were found between the three models in terms of accuracy ( p = 0.927), potential clinical risk ( p = 0.325), and ethical appropriateness ( p = 0.060). Accuracy scores were high in all models (≈ 4.2/5), while potential clinical risk scores were concentrated in the low range (≈ 1.05–1.70). Ethical appropriateness scores were generally high (≈ 4.3–4.5). Inter-observer agreement was generally moderate-good, higher in the emergency and special circumstances categories, and lower in the pediatric and adult orthodontics categories. ChatGPT, Gemini, and Copilot produce similar and highly accurate responses in orthodontic patient consultation. However, the fact that potential clinical risks have not been completely eliminated and the variability in ethical guidelines highlight the need for these systems to be used as a support tool under physician supervision, rather than as independent clinical decision-making tools. Patient safety, ethical frameworks, and user behavior must be considered in the clinical integration of AI-based systems.