Answers are grounded in this paper and cite their sources.
Evaluates AI's accuracy in automating complication assessment in surgeries, suggesting clinical efficiency.
Background The Comprehensive Complication Index (CCI) is a validated, clinically established, and sensitive measure of postoperative morbidity but is underutilised due to the labor-intensive and error-prone nature of manual complication extraction and grading. Large language models (LLMs) can analyse free-text surgical discharge summaries and accurately apply the Clavien-Dindo Classification (CDC). However, their ability to reliably compute the more complex CCI – requiring identification of multiple events, correct severity grading and weighted aggregation – remains unclear. Aims To evaluate the feasibility and accuracy of contemporary LLMs in extracting postoperative complications and computing the CCI from unstructured surgical discharge summaries. Methods Six LLMs (ChatGPT-5.2, Claude Sonnet 4.5, DeepSeek-V3.2, Mistral AI (2024), Gemini 3 Flash and Llama 4) were assessed using a tiered validation framework. After conceptual testing, each model analysed 20 de-identified real-world surgical discharge summaries. A three-layered prompting framework (preprocessing, complication extraction and computation, and consistency checking) was compared with naïve end-to-end analysis. Agreement with expert reference CCI values was assessed using intraclass correlation coefficients (ICC) and Bland-Altman analysis. Results All models correctly defined the CCI concept. Naïve analysis showed moderate agreement with reference CCI values (ICC(3,2) = 0.87), with the observed disagreement being driven by missed complications and incorrect aggregation. Structured prompting improved agreement to excellent (ICC(3,1)=0.93) with minimal systematic bias. ChatGPT and Claude achieved full agreement with reference CCI values across all cases. Discrepancies in other LLMs were attributable to ambiguity in clinical documentation or inherent interpretive boundaries of the CCI framework, rather than computational errors. Conclusion LLMs can accurately extract complications, assign CDC grades, and compute CCI from unstructured free-text surgical discharge summaries when guided by structured prompting. This demonstrates that AI can reliably perform complex morbidity aggregation and may reduce clinical workload while improving standardization of surgical outcome reporting. Further large-scale validation is warranted.
No takes yet. Share an insight, caveat, or question.
Borgas et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: