Key result
AI models achieve moderate-to-substantial agreement with clinicians in assigning ASA physical status classifications.
Why the study?
ASA physical status classification is limited by subjectivity and moderate inter-rater reliability, and the magnitude and reliability of AI models agreement with clinicians remained unclear.
Does artificial intelligence (AI) models improve inter-rater agreement in preoperative cases?
Meta-Analysis (n=402,336)
Does artificial intelligence (AI) models improve inter-rater agreement in preoperative cases?
Effect estimate: QWK 0.69 (95% CI 0.62-0.76)
AI systems, particularly LLMs, demonstrated moderate-to-substantial concordance with clinicians in assigning ASA physical status classifications, suggesting potential utility as reliable adjuncts for preoperative risk stratification.
Supports AI as adjunct for preoperative ASA classification; extends evidence for LLMs over traditional models in diagnostic agreement.
BACKGROUND: The American Society of Anesthesiologists (ASA) physical status classification remains the cornerstone of preoperative risk assessment but is limited by its subjective nature and only moderate inter-rater reliability. Artificial intelligence (AI) models have recently been applied to automate ASA prediction; however, the magnitude and reliability of their agreement with that of clinicians remain unclear. This systematic review and meta-analysis aimed to quantitatively synthesize the current evidence regarding the diagnostic performance of AI-based ASA classification systems. METHODS: A comprehensive search of the PubMed, Embase, Scopus, Web of Science, and CENTRAL (from inception to October 2025) databases was conducted. Eligible studies compared AI-predicted and clinician-assigned ASA classifications in real or simulated preoperative datasets. Study quality was assessed using the PROBAST+AI tool. The certainty of the evidence was graded using the GRADE framework. The primary outcome was inter-rater agreement measured using the quadratic weighted kappa (QWK); secondary outcomes included accuracy, sensitivity, and specificity. Random-effects meta-analysis and meta-regression analyses were performed. RESULTS: Thirteen studies involving 402,336 cases were analyzed. The pooled QWK was 0.69 (95% CI 0.62-0.76), representing moderate-to-substantial agreement with human assessors. Pooled accuracy was 0.66, sensitivity 0.51, and specificity 0.78. Subgroup analysis showed significantly higher agreement for LLMs than for traditional ML models (P=0.04). Meta-regression revealed lower performance in studies combining adult and pediatric data. CONCLUSIONS: This meta-analysis demonstrated that AI systems, particularly LLMs, showed moderate-to-substantial concordance with clinicians in assigning ASA physical status classifications, suggesting their potential utility as reliable adjuncts for preoperative risk stratification.
No takes yet. Share an insight, caveat, or question.
DOST et al. (2026) conducted a meta-analysis in Preoperative risk assessment (n=402,336). Artificial intelligence (AI) models vs. Clinician-assigned ASA classifications was evaluated on Inter-rater agreement measured using the quadratic weighted kappa (QWK) (QWK 0.69, 95% CI 0.62-0.76). Artificial intelligence models demonstrated moderate-to-substantial agreement with clinicians in assigning ASA physical status classifications (pooled QWK 0.69; 95% CI 0.62-0.76).
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: