Development and validation of a machine learning-based predictive model for carotid plaque in type 2 diabetes

Figures

• 

  Figure 1. Flowchart of machine learning model development for predicting carotid plaque in patients with type 2 diabetes. The schematic outlines the key stages from data collection to model deployment. Data Collection and Feature Engineering: Initial cohort included 2,292 patients. After excluding four patients, 2,288 were eligible. Exploratory data analysis (EDA), data preprocessing, feature derivation, and feature selection were performed to prepare the feature set. Data Splitting: The research population was divided into Model Development and Validation Data (n = 1,601), and independent Test Data (n = 687). Model Development: The development data were further split into training (70%) and validation (30%) sets. Hyperparameter tuning was conducted via a grid search and manual adjustment. Model Evaluation: The optimal model was evaluated using a test cohort. Performance metrics included the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), F1 score, accuracy, sensitivity, and specificity. Model Interpretation and Deployment: Model predictions were interpreted using the (SHAP). A web-based tool has been developed for clinical use.

• 

  Figure 2. Lasso regression-based feature selection. (A) Variation characteristics of variable coefficients; (B) The process of selecting the optimal value of the parameter λ in the lasso regression model is carried out by the cross-validation method.

• 

  Figure 3. Performance comparison of the ML models of benchmarking on the development/validation sets. (A) Receiver operating characteristic curve (ROC). (B) Precision-recall curve (PRC). (C) Calibration curve (CAL). (D) Decision curve analysis (DCA). AdaBoostM1(Adaptive Boosting M1), Gradient Boosting Machine (gbm), light-gradient boosting machine (lightgbm), Logistic Regression (logistic), Naïve Bayes(naive_ Bayes), Random Forest (ranger), and extreme Gradient Boosting (xgboost).

• 

  Figure 4. Performance heatmap of machine learning model benchmarking on development/validation sets. The models evaluated include AdaBoostM1(Adaptive Boosting M1), Gradient Boosting Machine (gbm), Light Gradient Boosting Machine (lightgbm), Logistic Regression (logistic), Naïve Bayes(naive_Bayes), Random Forest (ranger), and extreme Gradient Boosting (XGBoost). Performance metrics: Area Under the Curve (AUC), accuracy (ACC), Brier Score (Brier score), F-Beta (F-Beta Score), Precision-Recall Area Under the Curve (PRAUC), sensitivity, specificity, Negative Predictive Value (NPV), and Positive Predictive Value (PPV).

• 

  Figure 5. Performance of the logistic regression in the development/validation sets. (A) Receiver operating characteristic curve (ROC). (B) Confusion matrix. (C) Precision-recall curve (PRC). (D) Calibration curve (CAL). (E) Decision curve analysis (DCA). (F) Metric radar plot.

• 

  Figure 6. Performance of the logistic regression in the test set. (A) Receiver operating characteristic curve (ROC). (B) Confusion matrix. (C) Precision-recall curve (PRC). (D) Calibration curve (CAL). (E) Decision curve analysis (DCA).

• 

  Figure 7. Performance of the logistic regression in the temporal validation set. (A) Receiver operating characteristic curve (ROC). (B) Confusion matrix. (C) Precision-recall curve (PRC). (D) Calibration curve (CAL). (E) Decision curve analysis (DCA).

• 

  Figure 8. Model performance at the optimized threshold of 0.7 in temporal validation data. (A) Receiver operating characteristic (ROC) curve with area under the curve (AUC) of 0.758 (95% CI: 0.670–0.843). (B) Confusion matrix at threshold 0.7, showing 20 true negatives, 25 false positives, 11 false negatives, and 79 true positives. (C) Precision-recall (PR) curve with PR-AUC of 0.81 and baseline of 0.667. (D) Decision curve analysis demonstrating net benefit across threshold probabilities; the model (red curve) shows positive net benefit over treat-all (grey) and treat-none (black) strategies between threshold probabilities of approximately 0.10 and 0.70. (E) Calibration plot comparing predicted vs. observed event rates with 95% confidence bands (grey shading).

• 

  Figure 9. Global model explanation by the SHAP method. (A) SHAP summary dot plot of the Logistic Regression model. Each line represents a feature, and the x-axis represents SHAP values, indicating the impact of the feature on the outcome. Each dot represents an individual patient. The yellower the color, the larger the feature value; the bluer the color, the smaller the feature value. (B) SHAP feature importance plot of the Logistic Regression model.

• 

  Figure 10. Local model explanation by the SHAP method. Waterfall plot and evolution of risks contributed by each feature for individual patients at low (A) or high (B) risk of developing carotid plaque: (A) represents the individual patients toward the “non-carotid plaque” class, and (B) represent the individual patients toward the “carotid plaque” class. SHAP force plots of (C) patient 1 (true negative) and (D) patient 2 (true-positive). Each patient is represented on the x-axis, while the features’ contributions are represented on the y-axis; an increased yellow part for each individual patient represents a greater probability toward the decision of “carotid plaque”.

• 

  Figure 11. Online computing application presentation of the optimal logistic regression model.