Framingham Risk Score Prediction at 12 Months in the STANDFIRM Randomized Control Trial

Figures

• 

  Figure 1. Model fitting: Plot of ground truth vs machine learning predicted values. The tuning time to optimize each machine learning method is shown on the panel. It was longest for MLP and shortest for SVR. There was deviation of the predicted from observed FRS values with random forest, XGBoost and KNNcompared with SVR and MLP. The tuning time to optimize each machine learning method is shown on the panel. It was longest for MLP and shortest for SVR. There was deviation of the predicted from observed FRS values with random forest, XGBoost, and KNN compared with SVR and MLP. FRS indicates Framingham risk score; KNN, K‐nearest neighbor; MLP, multilayer perceptron; SVR, support vector regression; and XGBoost, extreme gradient boost.

• 

  Figure 2. Global interpretability: SHAP values and covariates for SVR: The SHAP plot show that the covariates with the largest SHAP values are nonmodifiable. The mean absolute SHAP values are displayed adjacent to the covariates for Catboost (A) and SVR (B). Following Catboost, FRS shift the prediction by 8.42 of 12.12 (69.5%), age by 1.58 (13.0%), male sex by 1.05 (8.7%). Following SVR, FRS shift prediction by 9.6 of a total of 14.39 points (66.7%), age by 1.04 points (7.2%), systolic blood pressure by 1.03 points (7.2%), male sex by 0.61 points (4.2%). The dimensionless SHAP value is on the x axis, and the y axis represent the covariates ordered from highest to lowest importance. The color appears as a range for continuous scale and as 2 colors for binary values. AF indicates atrial fibrillation; BMI, body mass index; Catboost, category boosting; eGFR, estimated glomerular filtration rate; FRS, Framingham risk score; HbA1c, glycosylated hemoglobin; IHD, ischemic heart disease; SHAP, Shapley additive explanation; and SVR, support vector regression.

• 

  Figure 3. Global interpretability: Sensitivity analysis at correlation coefficient threshold of 0.5. The R 2 remains stable at 0.707 as threshold 0.50 applied to correlation matrix. The original R 2 was 0.712. AF indicates atrial fibrillation; BMI, body mass index; Catboost, category boosting; FRS, Framingham risk score; HbA1c, glycosylated hemoglobin; IHD, ischemic heart disease; and SHAP, Shapley additive explanation.

• 

  Figure 4. Local interpretability: Waterfall plot showing individual patient prediction in test data. Illustrates the prediction for individual patients with >15‐point difference in predicted and observed FRS values in the first column and low difference in the second column. The expected values (23.829) from the model for each patient is provided below the x axis, and the observed value is at the top. The plot shows how the covariates move toward the prediction for that patient on the basis of the SHAP values of each covariate. A feature of this waterfall plot is that the covariates with the largest contribution are listed at the top and the ones with the least contribution are listed at the bottom. The direction of the shift provided with the red color implies the shift toward higher FRS and the blue color for shift toward the negative direction. AF indicates atrial fibrillation; BMI, body mass index; eGFR, estimated glomerular filtration rate; FRS, Framingham risk score; and SHAP, Shapley additive explanation.