This study aimed to evaluate the performance of seven AI-based chatbots (ChatGPT-4, ChatGPT-3.5, ChatGPT 01-Preview, ChatGPT 01-Mini, Microsoft Bing, Claude, and Google Gemini) in answering multiple-choice questions related to prosthetic dentistry from the Turkish Dental Specialization Mock Exam (DUSDATA TR). Additionally, the study investigated whether these chatbots could provide responses at an accuracy level comparable to general practitioners. A total of ten multiple-choice questions related to prosthetic dentistry were selected from a preparatory exam by a private educational institution. Two groups were formed: (1) General practitioners (Human Group, N = 657) and (2) AI-based chatbots. Each question was manually input into the chatbots, and their responses were recorded. Correct responses were marked as "1" and incorrect responses as "0". The consistency and accuracy of chatbot responses were analyzed using Fisher's exact test and Cochran's Q test. Statistical significance was set at p 0.05). However, Bing was identified as the chatbot with the highest number of incorrect responses. The study findings indicate that The performance of AI-based chatbots varies significantly and lacks consistency in answering prosthetic dentistry-related exam questions, necessitating further improvement before implementation.
Bulut et al. (Thu,) studied this question.