Key result
Mixed-effects XGBoost improves 30-day mortality prediction over fixed-effects models, especially in small datasets.
Why the study?
Ensemble tree-based models like Xgboost have a limited ability to handle correlated data such as hospital-level effects in cardiovascular risk prediction.
Does a binary-outcome mixed-effects Xgboost model improve prediction of 30-day mortality in patients undergoing cardiac surgery compared to standard fixed-effects machine learning and regression models?
Does a binary-outcome mixed-effects Xgboost model improve prediction of 30-day mortality in patients undergoing cardiac surgery compared to standard fixed-effects machine learning and regression models?
Integrating hospital-level random effects into Xgboost machine learning models significantly improves cardiovascular risk prediction when clinical dataset sample sizes are small.
Mixed-effects XGBoost may aid mortality prediction in small cardiac surgery cohorts; hypothesis-generating and requires prospective validation before practice change.
BACKGROUND: Ensemble tree-based models such as Xgboost are highly prognostic in cardiovascular medicine, as measured by the Clinical Effectiveness Metric (CEM). However, their ability to handle correlated data, such as hospital-level effects, is limited. OBJECTIVES: The aim of this work is to develop a binary-outcome mixed-effects Xgboost (BME) model that integrates random effects at the hospital level. To ascertain how well the model handles correlated data in cardiovascular outcomes, we aim to assess its performance and compare it to fixed-effects Xgboost and traditional logistic regression models. METHODS: A total of 227,087 patients over 17 years of age, undergoing cardiac surgery from 42 UK hospitals between 1 January 2012 and 31 March 2019, were included. The dataset was split into two cohorts: training/validation (n = 157,196; 2012-2016) and holdout (n = 69,891; 2017-2019). The outcome variable was 30-day mortality with hospitals considered as the clustering variable. The logistic regression, mixed-effects logistic regression, Xgboost and binary-outcome mixed-effects Xgboost (BME) were fitted to both standardized and unstandardized datasets across a range of sample sizes and the estimated prediction power metrics were compared to identify the best approach. RESULTS: The exploratory study found high variability in hospital-related mortality across datasets, which supported the adoption of the mixed-effects models. Unstandardized Xgboost BME demonstrated marked improvements in prediction power over the Xgboost model at small sample size ranges, but performance differences decreased as dataset sizes increased. Generalized linear models (glms) and generalized linear mixed-effects models (glmers) followed similar results, with the Xgboost models also excelling at greater sample sizes. CONCLUSIONS: These findings suggest that integrating mixed effects into machine learning models can enhance their performance on datasets where the sample size is small.
No takes yet. Share an insight, caveat, or question.
Dong et al. (2024) studied Patients undergoing cardiac surgery (n=227,087). Binary-outcome mixed-effects Xgboost (BME) model vs. Fixed-effects Xgboost (NC) and traditional logistic regression models (glm, glmer) was evaluated on Prediction power for in-hospital 30-day mortality (measured by Clinical Effectiveness Metric [CEM] and AUC). The binary-outcome mixed-effects Xgboost model demonstrated marked improvements in mortality prediction power over the fixed-effects model at small sample sizes, with performance converging as dataset size increased.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: