Key result
The TransferGBM model consistently outperformed transported and refitted models for predicting hospital-acquired acute kidney injury, achieving an average AUROC gain of 0.04 compared to refitted models.
Why the study?
Clinical prediction models experience performance drift as patient populations shift over time, creating a need for model updating frameworks that effectively utilize both old and new data.
Does TransferGBM improve the prediction of hospital-acquired acute kidney injury compared to transported or refitted baseline models in adult inpatients?
Cohort (n=141,696)
No
Does TransferGBM improve the prediction of hospital-acquired acute kidney injury compared to transported or refitted baseline models in adult inpatients?
Effect estimate: AUROC gain 0.04
A transfer learning approach using a stacking ensemble of gradient boosting machines effectively corrects temporal performance drift in clinical prediction models for acute kidney injury.
May mitigate temporal drift in AKI prediction models; hypothesis-generating and requires prospective validation before practice change.
BACKGROUND: Clinical prediction models suffer from performance drift as the patient population shifts over time. There is a great need for model updating approaches or modeling frameworks that can effectively use the old and new data. OBJECTIVE: Based on the paradigm of transfer learning, we aimed to develop a novel modeling framework that transfers old knowledge to the new environment for prediction tasks, and contributes to performance drift correction. METHODS: The proposed predictive modeling framework maintains a logistic regression-based stacking ensemble of 2 gradient boosting machine (GBM) models representing old and new knowledge learned from old and new data, respectively (referred to as transfer learning gradient boosting machine [TransferGBM]). The ensemble learning procedure can dynamically balance the old and new knowledge. Using 2010-2017 electronic health record data on a retrospective cohort of 141,696 patients, we validated TransferGBM for hospital-acquired acute kidney injury prediction. RESULTS: The baseline models (ie, transported models) that were trained on 2010 and 2011 data showed significant performance drift in the temporal validation with 2012-2017 data. Refitting these models using updated samples resulted in performance gains in nearly all cases. The proposed TransferGBM model succeeded in achieving uniformly better performance than the refitted models. CONCLUSIONS: Under the scenario of population shift, incorporating new knowledge while preserving old knowledge is essential for maintaining stable performance. Transfer learning combined with stacking ensemble learning can help achieve a balance of old and new knowledge in a flexible and adaptive way, even in the case of insufficient new data.
No takes yet. Share an insight, caveat, or question.
Zhang et al. (2022) conducted a cohort in Hospital-acquired acute kidney injury (AKI) (n=141,696). Transfer learning gradient boosting machine (TransferGBM) vs. TransportedGBM and RefittedGBM was evaluated on Area under the receiver operating characteristic curve (AUROC) for any AKI prediction (AUROC gain 0.04). The TransferGBM model consistently outperformed transported and refitted models for predicting hospital-acquired acute kidney injury, achieving an average AUROC gain of 0.04 compared to refitted models.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: