Abstract Introduction The Sequential Organ Failure Assessment (SOFA) categorizes organ system failure using common measurements of critical illness. This approach ignores interactions between different organ systems which are known to be clinically significant in predicting ICU outcomes. SOFA is known to have poor accuracy early on in critical illness. We hypothesized that standard multivariable machine learning (ML) prediction modeling approaches that account for interactions between variables would have better ICU mortality prediction accuracy than SOFA alone. Methods We collected data from the University of Chicago Medicine (UCMC), MIMIC IV, and Rush University Medical Center from 2018-2023 and transformed it into the Common Longitudinal ICU Data Format (CLIF). We identified adults who were on life support (defined as vasoactives or advanced respiratory support) for at least 6 consecutive hours. Using SOFA variables as covariates, we trained a SOFA-ML model for in-hospital mortality on the UCMC and MIMIC-IV datasets. In the training datasets, we compared the performance of 4 different ML methods (logistic regression, generalized additive models, elastic net, and LightGBM), ensembling the best model at both training sites into a final SOFA-ML model. We also compared different time periods of SOFA variable collection: measurements taken 42 hours before life support was initiated versus the full 48-hour window. We evaluated the statistical discrimination of SOFA and the final ensemble SOFA-ML model externally at Rush University Medical Center using area under the receiver operating characteristic curve (AUC). Results The study cohort included 33,520 patients from MIMIC-IV (65% white, median SOFA 2.5 IQR 2-4), 17,248 from UCMC (30% white, median SOFA 3.5 IQR: 2.5-5.0, and 17,971 from Rush (41% white, median SOFA 3 IQR 2-4). At both training sites, the best-trained model used LightGBM on a feature set that included the full 48-hour window. Across all feature sets, LightGBM outperformed logistic regression, generalized additive models, and elastic net. Across all models, the inclusion of the 6-hour period following life support initiation resulted in improved predictive performance. Conclusions Standard multivariable ML prediction models using SOFA variables improve prediction of ICU mortality compared to SOFA alone. These models capture interactions between different SOFA variables, a practice that intensive care physicians employ in patient practice. In addition, the improved performance as a result of including the 6-hour period after initiating life support highlights how critical the first few hours of life support are for the progression of critical illness and mortality. This abstract is funded by: None
Diaz et al. (Fri,) studied this question.