Development of a machine learning-derived model to predict unplanned ICU admissions after major non-cardiac surgery

BACKGROUND: Unplanned postoperative intensive care unit admissions (UIAs) are rare events that cause significant challenges to perioperative workflow. We describe the development of a machine-learning derived model to predict UIAs using only widely used preoperative variables. METHODS: This was a 3-year retrospective review of all adult surgeries under the General, Vascular, and Thoracic surgical services with anticipated length of greater than 180 minutes at a single institution. A UIA was defined as any post-operative patient recovering in the post-anesthesia care unit (PACU) requiring direct transfer to the intensive care unit (ICU) for higher level of care. We developed our prediction model with a gradient-boosting decision tree algorithm (XGBoost). The model incorporated sixteen generalizable predictor variables that were derived from the demographics and surgical booking details. Validation and evaluation were performed with 10-fold cross validation, and model performance was evaluated using the area under the receiver operating characteristic (ROC) curve, sensitivity, specificity, and likelihood ratio. RESULTS: A total of 4658 patients were included for analysis. The incidence of UIAs was 2.3%. With 10-fold cross validation, the area under the ROC curve was 0.80 (95% CI 0.74-0.86). Two decision thresholds were used, which achieved the best specificity of 94% (95% CI 92-96%), best positive likelihood ratio of 4.22 (95% CI 0.99-8.79), and best sensitivity of 82% (95% CI 58-100%). CONCLUSIONS: Our machine learning-derived model is a reliable tool for the perioperative clinician to predict a rare outcome in high-risk patients using only preoperative variables. Future studies will include prospective validation of this model at other institutions and real-time incorporation for improvement in perioperative workflow.