Developing an ML-Based Pretest Probability Model of Obstructive CAD in Patients With Stable Chest Pain

Figures

• 

• 

  Figure 1. Flowchart of the Study ACS = acute coronary syndrome; BMI = body mass index; CAD = coronary artery disease; C-STRAT Score = Pretest probability model of Coronary Plaques Study from Chinese Registry on Early Detection and Risk Stratification; CTA = computed tomography angiography; RF-CL = risk factor-weighted model; ESC2019 = model supported by the European Society of Cardiology.

• 

  Figure 2. Global Interpretation of Risk Predictor Contributions in C-STRAT Score The Shapley Additive exPlanations (SHAP) values (x-axis) represent the contribution of the outcome predictor (y-axis). Higher Shapley additive explanations values indicate a greater weight of the predictor in the model predictions. FH = family history; PVD = peripheral vascular disease; other abbreviations as in Figure 1.

• 

  Figure 3. ROC Curve Analysis for Different Models in the Training Data Set Compared with ESC2019 based on the type of chest pain, age, and male sex as predictive variables, the addition of risk factors including family history of CAD, smoking, hyperlipidemia, hypertension, and diabetes improved the model differentiation (RF-CL), and the model based on logistic regression analysis including the same variables as C-STRAT Score (LOGISTIC), the introduction of both clinical variables and machine learning algorithm (C-STRAT Score) further improved the model differentiation in train data set. Abbreviations as in Figure 1.

• 

  Figure 4. ROC Curve Analysis for Different Models in the Testing Data Set Compared with ESC2019 based on the type of chest pain, age, and male sex as predictive variables, the addition of risk factors including family history of CAD, smoking, hyperlipidemia, hypertension, and diabetes improved the model differentiation (RF-CL), and the model based on logistic regression analysis including the same variables as C-STRAT Score (LOGISTIC), the introduction of both clinical variables and machine learning algorithm (C-STRAT Score) further improved the model differentiation in test data set. Abbreviations as in Figure 1.

• 

  Figure 5. Calibration Curve for Different Models in the Training Data Set The x-axis represents the predicted probability and the y-axis denotes the actual proportion of obstructive CAD. The diagonal dotted line indicates the best prediction by an ideal model. This analysis details the results about the predictive accuracy of C-STRAT score, LOGISTIC, RF-CL, and ESC2019. C-STRAT score from machine learning algorithm obtained a fairly satisfactory calibration, whereas the other models (RF-CL, ESC2019) calibrated poorly. Abbreviations as in Figure 1.

• 

  Figure 6. Calibration Curve for Different Models in the Testing Data Set The x-axis represents the predicted probability and the y-axis denotes the actual proportion of obstructive CAD. The diagonal dotted line indicates the best prediction by an ideal model. This analysis details the results about the predictive accuracy of C-STRAT score, LOGISTIC, RF-CL, and ESC2019. C-STRAT score from machine learning algorithm obtained a fairly satisfactory calibration while the other models (RF-CL, ESC2019) calibrated poorly. Abbreviations as in Figure 1.

• 

  Figure 7. Decision Curve for Different Models in the Testing Data Set The net benefits were measured at different threshold probabilities. The gray line represents the assumption that all patients have obstructive CAD. The horizontal black line represents the assumption that no patients have obstructive CAD. C-STRAT score from machine learning algorithm provided positive benefits for patients at any probability cutoff value, whereas the other 3 models benefited poorly. Abbreviations as in Figure 1.

• 

  Central Illustration. Flowchart of Building and Validating an Innovative Pretest Probability Model This figure demonstrates the building and validation of machine learning models (C-STRAT Score [pretest probability model of Coronary Plaques Study From Chinese Registry on Early Detection and Risk Stratification]) against the other pretest probability (PTP) models (LOGISTIC [conventional logistic regression model], RF-CL [risk factor-weighted model], ESC2019 [model supported by the European Society of Cardiology]). C-STRAT score achieved the highest area under the receiver-operating characteristic curve (AUC) (AUC: 0.769; 95% CI: 0.753-0.784) than other pretest probability model (LOGISTIC, AUC: 0.744; 95% CI: 0.728-0.761; P < 0.001; RF-CL, AUC: 0.743; 95% CI: 0.727-0.760; P < 0.001; ESC2019, AUC: 0.725; 95% CI: 0.708-0.742; P < 0.001). The results indicate that a high-performance pretest probability model derived from machine learning (ML) algorithm, was developed based on a multicenter Chinese population and expected to facilitate the decision making for downstream tests. BMI = body mass index; CAD = coronary artery disease; PVD = peripheral vascular disease.