Machine learning models marginally outperformed traditional risk scores in the discrimination of MACE (C-statistic 0.83 vs 0.71; P=0.007) and major bleeding following PCI.
Meta-Analysis (n=4,105,916)
Do machine learning models improve risk prediction for clinical outcomes compared to traditional statistical methods in patients undergoing PCI?
Machine learning models marginally outperform traditional risk scores in discriminating MACE and major bleeding following PCI, though significant methodological and validation limitations remain before clinical implementation.
Effect estimate: C-statistic (95% CI 0.84-0.91)
Absolute Event Rate: 0.89% vs 0.86%
p-value: p=0.54
Abstract Aims Accurate prediction of clinical outcomes following percutaneous coronary intervention (PCI) is essential for mitigating risk and peri-procedural planning. Traditional risk models have demonstrated a modest predictive value. Machine learning (ML) models offer an alternative risk stratification that may provide improved predictive accuracy. Methods and results This study was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses, Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies and Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis guidelines. PubMed, EMBASE, Web of Science, and Cochrane databases were searched until 1 November 2023 for studies comparing ML models with traditional statistical methods for event prediction after PCI. The primary outcome was comparative discrimination measured by C-statistics with 95% confidence intervals (CIs) between ML models and traditional methods in estimating the risk of all-cause mortality, major bleeding, and the composite outcome major adverse cardiovascular events (MACE). Thirty-four models were included across 13 observational studies (4 105 916 patients). For all-cause mortality, the pooled C-statistic for top-performing ML models was 0.89 (95%CI, 0.84–0.91), compared with 0.86 (95% CI, 0.80–0.93) for traditional methods (P = 0.54). For major bleeding, the pooled C-statistic for ML models was 0.80 (95% CI, 0.77–0.84), compared with 0.78 (95% CI, 0.77–0.79) for traditional methods (P = 0.02). For MACE, the C-statistic for ML models was 0.83 (95% CI, 0.75–0.91), compared with 0.71 (95% CI, 0.69–0.74) for traditional methods (P = 0.007). Out of all included models, only one model was externally validated. Calibration was inconsistently reported across all models. Prediction Model Risk of Bias Assessment Tool demonstrated a high risk of bias across all studies. Conclusion Machine learning models marginally outperformed traditional risk scores in the discrimination of MACE and major bleeding following PCI. While integration of ML algorithms into electronic healthcare systems has been hypothesized to improve peri-procedural risk stratification, immediate implementation in the clinical setting remains uncertain. Further research is required to overcome methodological and validation limitations.
Zaka et al. (Mon,) conducted a meta-analysis in percutaneous coronary intervention (PCI) (n=4,105,916). Machine learning models vs. Traditional statistical methods was evaluated on Comparative discrimination measured by C-statistics for all-cause mortality (C-statistic, 95% CI 0.84-0.91, p=0.54). Machine learning models marginally outperformed traditional risk scores in the discrimination of MACE (C-statistic 0.83 vs 0.71; P=0.007) and major bleeding following PCI.