Key result
Concordant ML and STS predictions improve post-op renal failure risk discrimination to 0.81 C-index.
Why the study?
The role of supplementing Society of Thoracic Surgeons risk models for surgical aortic valve replacement with machine learning to improve risk prediction was evaluated.
Does supplementing STS risk models with machine learning improve risk prediction in adults undergoing surgical aortic valve replacement?
Observational (n=243,142)
Yes
Does supplementing STS risk models with machine learning improve risk prediction in adults undergoing surgical aortic valve replacement?
Supplementing existing STS risk models with machine learning identifies patient subsets with discordant risk estimates where predictive performance declines, suggesting vulnerability in current models.
ML-STS risk concordance improves SAVR prediction; leaves open whether supplementation aids decisions without prospective validation.
Background This study evaluated the role of supplementing Society of Thoracic Surgeons (STS) risk models for surgical aortic valve replacement with machine learning (ML). Methods and Results Adults undergoing isolated surgical aortic valve replacement in the STS National Database between 2007 and 2017 were included. ML models for operative mortality and major morbidity were previously developed using extreme gradient boosting. Concordance and discordance in predicted risk between ML and STS models were defined using equal‐size tertile‐based thresholds of risk. Calibration metrics and discriminatory capability were compared between concordant and discordant patients. A total of 243 142 patients were included. Nearly all calibration metrics were improved in cases of concordance. Similarly, concordance indices improved substantially in cases of concordance for all models with the exception of deep sternal wound infection. The greatest improvements in concordant versus discordant cases were in renal failure: ML model (concordance index, 0.660 [95% CI, 0.632–0.687] discordant versus 0.808 [95% CI, 0.794–0.822] concordant) and STS model (concordance index, 0.573 [95% CI, 0.549–0.576] discordant versus 0.797 [95% CI, 0.782–0.811] concordant) (each P <0.001). Excluding deep sternal wound infection, the concordance indices ranged from 0.549 to 0.660 for discordant cases and 0.674 to 0.808 for concordant cases. Conclusions Supplementing ML models with existing STS models for surgical aortic valve replacement may have an important role in risk prediction and should be explored further. In particular, for the roughly 25% to 50% of patients demonstrating discordance in estimated risk between ML and STS, there appears to be a substantial decline in predictive performance suggesting vulnerability of the existing models in these patient subsets.
No takes yet. Share an insight, caveat, or question.
Kilic et al. (2021) conducted an observational in Surgical aortic valve replacement (n=243,142). Concordant risk prediction between ML and STS models vs. Discordant risk prediction between ML and STS models was evaluated on Predictive performance (concordance index) for renal failure using the ML model (p=<0.001). Concordant risk predictions between ML and STS models yielded significantly better predictive performance for renal failure than discordant predictions (C-index 0.808 vs 0.660; P<0.001).
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: