The TransferGBM model consistently outperformed transported and refitted models for predicting hospital-acquired acute kidney injury, achieving an average AUROC gain of 0.04 compared to refitted models.
Cohort (n=141,696)
No
Does TransferGBM improve the prediction of hospital-acquired acute kidney injury compared to transported or refitted baseline models in adult inpatients?
A transfer learning approach using a stacking ensemble of gradient boosting machines effectively corrects temporal performance drift in clinical prediction models for acute kidney injury.
Effect estimate: AUROC gain 0.04
BACKGROUND: Clinical prediction models suffer from performance drift as the patient population shifts over time. There is a great need for model updating approaches or modeling frameworks that can effectively use the old and new data. OBJECTIVE: Based on the paradigm of transfer learning, we aimed to develop a novel modeling framework that transfers old knowledge to the new environment for prediction tasks, and contributes to performance drift correction. METHODS: The proposed predictive modeling framework maintains a logistic regression-based stacking ensemble of 2 gradient boosting machine (GBM) models representing old and new knowledge learned from old and new data, respectively (referred to as transfer learning gradient boosting machine TransferGBM). The ensemble learning procedure can dynamically balance the old and new knowledge. Using 2010-2017 electronic health record data on a retrospective cohort of 141,696 patients, we validated TransferGBM for hospital-acquired acute kidney injury prediction. RESULTS: The baseline models (ie, transported models) that were trained on 2010 and 2011 data showed significant performance drift in the temporal validation with 2012-2017 data. Refitting these models using updated samples resulted in performance gains in nearly all cases. The proposed TransferGBM model succeeded in achieving uniformly better performance than the refitted models. CONCLUSIONS: Under the scenario of population shift, incorporating new knowledge while preserving old knowledge is essential for maintaining stable performance. Transfer learning combined with stacking ensemble learning can help achieve a balance of old and new knowledge in a flexible and adaptive way, even in the case of insufficient new data.
Zhang et al. (Wed,) conducted a cohort in Hospital-acquired acute kidney injury (AKI) (n=141,696). Transfer learning gradient boosting machine (TransferGBM) vs. TransportedGBM and RefittedGBM was evaluated on Area under the receiver operating characteristic curve (AUROC) for any AKI prediction (AUROC gain 0.04). The TransferGBM model consistently outperformed transported and refitted models for predicting hospital-acquired acute kidney injury, achieving an average AUROC gain of 0.04 compared to refitted models.