Key result
Machine learning models including CatBoost predicted one-year mortality after TAVI with an external validation AUC of 0.68, using a novel protocol that avoids patient data exchange.
Why the study?
Current TAVI risk scores do not yet leverage modern machine learning to improve one-year mortality risk stratification, while strict data-sharing regulations hinder external model validation.
Can machine learning models accurately predict one-year mortality in TAVI patients when externally validated without exchanging patient data?
Observational (n=1,931)
Yes
Can machine learning models accurately predict one-year mortality in TAVI patients when externally validated without exchanging patient data?
Effect estimate: AUC 0.68 for CatBoost external validation
Machine learning models can be robustly externally validated across TAVI centers without exchanging patient data, achieving moderate predictive accuracy for one-year mortality.
Current prognostic risk scores for transcatheter aortic valve implantation (TAVI) do not benefit yet from modern machine learning techniques, which can improve risk stratification of one-year mortality of patients before TAVI. Despite the advancement of machine learning in healthcare, data sharing regulations are very strict and typically prevent exchanging patient data, without the involvement of ethical committees. A very robust validation approach, including 1300 and 631 patients per center, was performed to validate a machine learning model of one center at the other external center with their data, in a mutual fashion. This was achieved without any data exchange but solely by exchanging the models and the data processing pipelines. A dedicated exchange protocol was designed to evaluate and quantify the model's robustness on the data of the external center. Models developed with the larger dataset offered similar or higher prediction accuracy on the external validation. Logistic regression, random forest and CatBoost lead to areas under curve of the ROC of 0.65, 0.67 and 0.65 for the internal validation and of 0.62, 0.66, 0.68 for the external validation, respectively. We propose a scalable exchange protocol which can be further extended on other TAVI centers, but more generally to any other clinical scenario, that could benefit from this validation approach.
No takes yet. Share an insight, caveat, or question.
Mamprin et al. (2021) conducted an observational in Transcatheter aortic valve implantation (TAVI) (n=1,931). Machine learning models (Logistic regression, random forest, CatBoost) was evaluated on Area under the ROC curve for predicting one-year mortality (AUC 0.68 for CatBoost external validation). Machine learning models including CatBoost predicted one-year mortality after TAVI with an external validation AUC of 0.68, using a novel protocol that avoids patient data exchange.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: