ABSTRACT Objective Artificial intelligence (AI)‐driven large language models, such as Chat Generative Pre‐Trained Transformer (ChatGPT)‐4 Plus, are increasingly used for patient education and clinical decision support in oral oncology, although their accuracy in oral cancer (OC) management remains uncertain. This study evaluates the accuracy of ChatGPT‐4 Plus responses to clinically relevant questions regarding OC diagnosis, treatment, recovery, and prevention. Methods A cross‐sectional study assessed 65 clinically relevant OC‐related questions using a paid ChatGPT‐4 Plus subscription without modifications. Three oral medicine specialists and one radiation oncologist rated accuracy on a four‐point scoring system. Interrater reliability was measured with the intraclass correlation coefficient (ICC), and chi‐square tests were used for comparisons. Results Among 65 questions, 63% of responses were Score 1, with none rated as Score 4. Score 1 was most frequent in Recovery (72%), followed by Treatment (62%), Prevention (60%), and Diagnosis (55%). Scores 2 and 3 responses were highest in Diagnosis (45%). Recovery had significantly higher Score 1 responses than Diagnosis ( p < 0.05), while other comparisons were not significant. ICC ranged from 0.85 to 0.93. Conclusions ChatGPT‐4 Plus provided accurate responses to clinically relevant OC‐related questions, particularly regarding recovery. However, diagnostic inconsistencies highlight the need for clinician oversight before integrating AI into practice.
Yılmaz et al. (Fri,) studied this question.