Multicenter development of an interpretable machine learning model for prognosis of acute pancreatitis with acute kidney injury

Background Patients with severe acute pancreatitis complicated by acute kidney injury (AP-AKI) face extremely high mortality rates; therefore, early and precise risk stratification is crucial for optimizing individualized treatment strategies. However, traditional linear prognostic models have inherent limitations in capturing the non-linear associations between complex clinical features and patient outcomes, and they often lack sufficient clinical generalizability. This study aimed to develop, externally validate, and deploy a highly accurate and interpretable machine learning model to predict the survival outcome of AP-AKI patients. Methods Early clinical data of AP-AKI patients were extracted from the large intensive care database MIMIC-IV, with all-cause mortality at 60 days serving as the primary observation endpoint. The study utilized three algorithms (Boruta, LASSO, and MRMR) to screen 18 core features, subsequently constructing and systematically comparing 24 machine learning models. The models underwent comprehensive evaluation through the Area Under the Receiver Operating Characteristic Curve (AUC), calibration metrics, and Decision Curve Analysis (DCA), while external validation was accomplished in the independent eICU database. Additionally, the SHAP method was applied to deeply clarify the complex non-linear interaction patterns among prognostic factors, and a convenient interactive online application was developed based on the optimal model. Results Among the 24 included models, the Random Forest (RF) model demonstrated the best holistic performance. It achieved an AUC of 0.8963 (0.8479, 0.9447) in the internal validation set. Despite facing extreme class imbalance and cross-center data heterogeneity in the eICU external validation set, the model maintained excellent discrimination with an AUC of 0.8122 (0.7139, 0.9106). Compared with numerous traditional and machine learning baseline models, the RF model exhibited optimal overall calibration, and DCA confirmed that it achieved highly valuable clinical net benefit across a wide range of decision thresholds. SHAP analysis profoundly revealed multiple non-linear driving patterns, such as step-wise or “S-shaped” threshold effects of the 18 core features (prominently including aki_kdigo_stage, charlson_comorbidity_index, rdw, ast, and sapsii_score) on mortality risk. The successfully deployed interactive Shiny web application ( https://medicalpredictor.shinyapps.io/Online_Prediction_of_60-Day_Survival_Status_in_Patients ) enables code-free individualized risk prediction and feature attribution interpretation. Conclusions This study successfully constructed and externally validated a highly interpretable RF model for the precise prediction of survival outcomes in AP-AKI patients. The online interactive application deployed based on this model provides a convenient, intuitive, and evidence-based decision support tool for early clinical risk stratification and individualized precision intervention. Future multi-center, prospective randomized controlled studies are urgently needed to further validate its real-world clinical utility, along with exploring the inclusion of time-series data to develop more comprehensive longitudinal dynamic predictive models.