Answers are grounded in this paper and cite their sources.
BackgroundFrontier large language models (LLMs) are emerging as perioperative clinical decision-support tools, but prospective comparisons with consultant-led multidisciplinary teams (MDTs) remain limited. We evaluated the inter-model agreement, concordance with consultant-led multidisciplinary care, safety, and assessment-level inter-model consensus of four frontier LLMs in elective gastrointestinal surgery within an Enhanced Recovery After Surgery (ERAS) pathway.MethodsIn this prospective longitudinal observational study, 250 adults contributed 2398 longitudinal clinical assessments and 9592 individual LLM recommendations from preoperative evaluation through discharge. Each assessment was independently evaluated by ChatGPT, Claude, Gemini, and DeepSeek using identical locked prompts in separate conversations without memory retention or response regeneration. Recommendations were compared with the consultant-led MDT clinical reference standard. Primary outcome was MDT agreement; secondary outcomes included Cohen's κ, domain-specific agreement, assessment-level inter-model consensus, hallucinations, unsafe recommendations, and longitudinal performance using mixed-effects regression with generalized estimating equation and bootstrap sensitivity analyses.ResultsA total of 9592 independent LLM evaluations were analyzed. Overall agreement was highest for Gemini (93.2%; κ = 0.87), followed by Claude (92.9%; κ = 0.86), ChatGPT (91.8%; κ = 0.84), and DeepSeek (88.8%; κ = 0.79). Complete inter-model consensus occurred in 1834 assessments (76.5%), majority consensus in 352 (14.7%), split consensus in 154 (6.4%), and complete disagreement in 58 (2.4%). Of 799 recommendations discordant with the MDT reference standard, 605 met the prespecified criteria for formal adjudication. An additional 95 recommendations were randomly selected from those not meeting the formal criteria, yielding 700 recommendations for blinded expert adjudication. Confirmed safety events were uncommon (2.6-6.0% of evaluations).ConclusionsFrontier LLMs demonstrated high concordance with consultant-led MDT recommendations under standardized prompting conditions. Greater assessment-level inter-model consensus was associated with higher MDT concordance. These findings support further prospective evaluation of LLMs as supervised adjunctive perioperative decision-support tools but do not establish clinical correctness, outcome benefit, or suitability for autonomous decision-making.
No takes yet. Share an insight, caveat, or question.
Kesharwani et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: