Clinical Evaluation of an AI-Assisted Decision Support System for General Anesthesia Management Based on Data From 6 Centers: Comparative Study

Background AI is rapidly transforming medical practice, with emerging applications in perioperative care and anesthesiology. However, the clinical implementation of AI-assisted decision-making systems in anesthetic management remains challenging and requires comprehensive evaluation. Objective This study aimed to assess the performance and clinical applicability of an AI-assisted decision support system (ZW-AA-001) for general anesthesia management by comparing its decisions with those of experienced anesthesiologists across 6 medical centers. Methods A multicenter retrospective study was conducted using perioperative data from 1008 patients who underwent elective noncardiac surgeries under total intravenous anesthesia. The AI system’s recommendations for anesthetic and hemodynamic medication adjustments were compared with anesthesiologists’ decisions. Key outcomes included decision concordance rates, temporal performance, and consistency across centers. Advanced statistical methods, including prevalence-adjusted and bias-adjusted κ (PABAK) and Gwet’s first-order agreement coefficient (AC1), were used to evaluate agreement metrics. Results The study included 1008 patients, with a median age of 50 (IQR 37-59) years and female predominance (619/1008, 61.4%). During anesthesia maintenance, the AI system demonstrated moderate overall decision agreement with anesthesiologists (73.3%, 95% CI 72.4%-74.3%). Analysis of center-specific data revealed generally consistent performance across all 6 centers. For propofol management, high concordance was observed in dosage adjustment decisions (91.1%, 95% CI 90.4%-91.8%), though fair agreement was found in adjustment direction (68.6%, 95% CI 67.3%-69.8%). The AI system showed significantly faster decision-making time compared to anesthesiologists for the adjustment of propofol (pseudomedian difference –77.5, 95% CI –79.5 to –75.5 seconds; P<.001). Although esmolol-related decisions showed a numerically higher concordance of 71.4%, this was not statistically significant (PABAK=0.429; P=.21; AC1=0.622; P=.09). Decisions for atropine, ephedrine, and urapidil demonstrated substantially lower agreement (17.4%-29.8%). The AI system recommended hemodynamic interventions more frequently than anesthesiologists across all medications. Conclusions The AI-assisted decision support system demonstrated varying levels of concordance with anesthesiologists in managing surgery, with the highest agreement observed in propofol administration. However, the system’s lower agreement in hemodynamic medication management highlights the need for further optimization and validation. These findings underscore the potential of AI systems to enhance anesthetic decision-making, improve efficiency, and address workforce challenges in anesthesiology. Future research should focus on expanding the system’s capabilities, validating its performance in prospective clinical trials, and ensuring its integration into diverse clinical settings.