Answers are grounded in this paper and cite their sources.
Background Effective patient education is essential in pediatric neurology but challenged by time constraints and variable health literacy. Large language models (LLMs) offer a potential solution, yet their accuracy and safety in this specialized field remain incompletely evaluated. Methods In a prospective, blinded benchmarking study, LLM-generated responses to ten standardized pediatric neurology prompts were evaluated. Responses were assessed for factual accuracy, safety, and perceived usability; the study did not assess real-world clinical decision support or caregiver comprehension. A blinded panel of seven consultant pediatric neurology evaluators assessed the outputs using an expert-developed rubric measuring factual accuracy, safety, clinical appropriateness, and usability. The unit of analysis was the scenario-model pair ( n = 40), with median scores across raters used for primary comparisons. Results Linear mixed-effects models revealed significant performance variation across models (F(3,27) = 8.76, p < 0.001). DeepSeek achieved the highest overall composite score (4.01 ± 0.61) and zero safety flags. ChatGPT scored moderately (3.72 ± 0.75) but accounted for most safety issues (5/8), including a contraindicated medication recommendation. Gemini and Perplexity scored lower (3.45 ± 0.77 and 3.38 ± 0.71). The overall composite difference between DeepSeek and ChatGPT was not statistically significant ( p = 0.087). DeepSeek had the highest System Usability Scale score (74.2/100). Presence of any consensus safety flag drastically reduced the odds of expert approval for caregiver-facing use after clinician review (OR: 0.08, p < 0.001). Conclusions DeepSeek produced no consensus safety flags in this benchmark and had the highest expert-rated usability score, but its overall composite score was not statistically different from ChatGPT. These findings are limited to the scenarios tested and do not establish broad safety, clinical effectiveness, or caregiver benefit. No caregivers were directly studied. No model is ready for autonomous use, and clinician verification remains mandatory. This benchmark provides a methodology for specialty-specific evaluation.
No takes yet. Share an insight, caveat, or question.
Muthaffar et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: