Accuracy, Completeness, Readability, and Safety of Large Language Model-Based Chatbots in Vital Pulp Therapy

Large language model (LLM)-based chatbots are increasingly used to access health-related information, but their reliability in vital pulp therapy (VPT) decision-making remains unclear. This study aimed to compare five current LLM-based chatbots in caries-associated VPT scenarios in terms of accuracy, completeness, readability, and predefined potentially harmful recommendations. Twenty hypothetical VPT scenarios representing key decision points in mature and immature permanent teeth were developed from current endodontic and paediatric dentistry guidelines. ChatGPT-5.2, Claude Opus 4.6, DeepSeek V3.2, Gemini 3 Pro, and Perplexity Pro were queried using a standardized prompt. Responses were independently scored by four expert evaluators using 5-point accuracy and 3-point completeness scales. Readability was assessed using five readability indices. Predefined harm flags were coded as binary variables. Statistical analyses included intraclass correlation coefficients, linear mixed-effects models with Bonferroni-adjusted pairwise comparisons, and the Fisher–Freeman–Halton exact test. Accuracy scores differed significantly among chatbots ( P = .001), with Perplexity Pro showing the highest score and DeepSeek V3.2 the lowest. Completeness scores also differed significantly ( P < .001), with Claude Opus 4.6 scoring highest and Gemini 3 Pro lowest. Apex maturation status had a significant main effect on completeness ( P = .049), with higher scores for immature permanent tooth scenarios, but not on accuracy ( P = .857). The chatbot platform × apex maturation interaction was significant for accuracy ( P = .037) but not completeness ( P = .267). All readability indices differed significantly among chatbots ( P < .001). Only two responses had predefined harm flags, both generated by DeepSeek V3.2, with no significant difference among platforms ( P = .192). Current LLM-based chatbots generally demonstrated high accuracy in VPT-related scenarios; however, relative accuracy performance varied according to apex maturation status, while completeness and readability also differed among chatbots. LLM outputs may support VPT-related information access but should be verified by clinicians before clinical or patient-facing use.