A machine learning-based model for predicting 28-day mortality in ICU patients identified through radiotherapy-related records: validation using public databases and a single-center cohort

Objective Critically ill patients with documented radiotherapy-related records represent a population in whom early short-term mortality assessment may be clinically relevant. We aimed to develop and validate an interpretable machine-learning model for predicting 28-day mortality in patients who remained observable through the first 24 h after ICU admission. Methods We included patients from MIMIC-IV for model training (n=908), the MIMIC-III CareVue subset v1.4 for temporal testing (n=499), and the eICU/local cohorts for external validation (n=118). Eligibility was determined solely by radiotherapy-related diagnosis or procedure records, which were not used as model predictors. Model predictors were derived primarily from information recorded during the first 24 h after ICU admission. Missing data were handled using multiple imputation. LASSO regression was used for feature selection in the training cohort. Seven machine-learning models were compared based on their discrimination, and the selected model was further evaluated using calibration, decision-curve analysis, and SHAP interpretation. A sensitivity analysis excluding sepsis and acute kidney injury was performed. Results The 28-day mortality rates were 11.56% (105/908), 11.62% (58/499), and 12.71% (15/118) in the training, temporal testing, and external validation cohorts, respectively. LASSO selected chloride, SOFA score, RDW, respiratory rate, sepsis, and acute kidney injury. In the temporal testing cohort, random forest achieved the highest observed AUC (0.868; 95% CI, 0.815-0.917) and showed greater discrimination than SOFA-based benchmarks, six-variable logistic regression, and ANN in paired comparisons. In the external validation cohort, random forest showed an AUC of 0.785 (95% CI, 0.650-0.909). After exclusion of sepsis and acute kidney injury, the observed AUC decreased to 0.843 in temporal testing and 0.713 in external validation; the paired confidence intervals for both AUC differences included zero. Conclusion A six-variable random forest model showed useful discrimination for 28-day mortality among ICU patients identified through radiotherapy-related records and maintained moderate discrimination in external validation. SHAP analysis enhanced model interpretability by illustrating the contribution of routinely available general ICU variables. Radiotherapy-related records served solely to define the study population; this study did not evaluate radiotherapy exposure or radiotherapy-specific mechanisms. These findings support further evaluation of the model for prognostic risk stratification in larger prospective cohorts.