Development and validation of a machine learning prediction model for complicated appendicitis in adults: a multicenter retrospective cohort study

Accurate differentiation between complicated appendicitis (CA) and uncomplicated appendicitis is clinically important in the nonoperative management of acute appendicitis in adults. However, multicenter evidence on risk stratification for CA remains limited. To develop and externally validate a machine learning model integrating routine clinical, laboratory, and computed tomography (CT) information for preoperative risk stratification of CA in adults with acute appendicitis. This multicenter retrospective cohort study included 1573 adults with acute appendicitis from 4 hospitals. A total of 652 patients from center 1 were used for model development and were randomly divided into a training cohort and an internal validation cohort at a 7:3 ratio; 438 patients from center 2 were used as external validation cohort 1; and 483 patients from centers 3 and 4 combined were used as external validation cohort 2. Models were developed using 10 machine learning algorithms and optimized by cross-validation. Discrimination, calibration, and clinical net benefit were assessed using the area under the receiver operating characteristic curve (AUC), Brier score, and decision curve analysis (DCA), respectively. The final model was interpreted using SHapley Additive exPlanations (SHAP), and a web-based online prediction platform was developed. Based only on cross-validation results from the training cohort and the prespecified model selection strategy, the support vector machine model was selected as the final model. The model achieved an AUC of 0.910 (95% CI, 0.840–0.968) in the internal validation cohort, 0.797 (95% CI, 0.743–0.846) in external validation cohort 1, and 0.829 (95% CI, 0.776–0.879) in external validation cohort 2. The model showed good calibration and clinical net benefit in internal validation. Substantial calibration drift was observed in both external validation cohorts; following intercept-only recalibration, the Brier scores decreased from 0.16 to 0.14 and from 0.14 to 0.12, respectively. SHAP analysis showed that fibrinogen, neutrophil-to-lymphocyte ratio, and pain duration were important predictors. The web-based online prediction platform developed from the final model provides individualized risk estimates. The model may support preoperative risk stratification for CA in adults, but the original, unrecalibrated model must not be applied directly to local populations. Local recalibration is mandatory before clinical deployment, and the recalibrated model still requires local validation and prospective evaluation of its clinical impact.