Abstract Rationale We previously developed the Predictive Risk Score (PRS) to identify patients at risk for critical illness using electronic health record (EHR) data. In this study, we sought to improve the PRS’s prediction of future intensive care unit (ICU) admission and survival by using machine learning methods. Methods We obtained EHR data for 48,127 adults aged 50 years or older seen by their primary care provider (PCP) at two large, urban, academic health systems between January 1 and December 31, 2017. We extracted demographics, laboratory values, diagnoses, medications, procedures, and healthcare utilization until two years after the index PCP visit. For the two subsequent years, subjects were classified as alive without ICU admission, ICU survivors, and deceased. We split the cohort into training (70%) and testing subsets (30%). A multinomial logistic regression model using previously identified predictors in the PRS served as a comparison. We compared the results from the XGBoost multiclass classifier and feed-forward neural network incorporating three hidden layers trained on 332 standardized features. Performance was evaluated using overall AUC, pairwise ROC curves, and confusion matrices. To interpret model behavior, SHapley Additive exPlanations (SHAP) were applied to both XGBoost and neural network models. Features were ranked by mean absolute SHAP value, and the top 20 predictors were identified as the most influential contributors to classification. Results In the overall sample of 48,127 patients, 60.8% were female, 32.9% were Black, and 6.2% experienced ICU admission with survival during the follow up period. Of the top twenty predictive features, eleven were common to both machine learning models: abnormal albumin; abnormal creatinine; abnormal low-density lipoprotein; abnormal red blood cell count; age; encounters for general examination; encounter for screening for malignant neoplasms; essential hypertension; overweight and obesity; other anxiety disorders; and other chronic obstructive pulmonary disease. XGBoost correctly flagged 47.6% of death cases (109/229) and 42.8% of ICU survival (384/898), while neural network correctly identified 30.6% of death cases (70/229) and 48.4% of ICU survival (435/898). XGBoost and neural network provided similar AUCs in differentiating ICU survivors vs. Alive without ICU admission (XGBoost: 0.73 (95%CI 0.71, 0.75), Neural Network: 0.70 (95%CI 0.68, 0.72)). Conclusions Our results suggest that machine learning models may be useful in the development of predictive risk models for ICU survival. Follow up studies utilizing prospective validation cohorts to confirm these findings are needed. This abstract is funded by: None
Vroegop et al. (Fri,) studied this question.