A machine learning model for automated anesthesia risk classification in lumbar spinal stenosis patients: development and multicenter validation

Figures

• 

  Figure 1. Workflow diagram of the LightGBM-based model development and validation process. A total of 700 patients were divided into training (n = 480), internal validation (n = 120), and external validation (n = 100) sets. The training set was used for feature selection, 5-fold cross-validation, and hyperparameter tuning (grid search + Bayesian optimization) with LightGBM. The internal and external validation sets were used exclusively for final performance evaluation and generalization assessment.

• 

  Figure 2. Learning curves of the LightGBM model. The learning curve shows that both the training loss (solid line) and the validation loss (dashed line) decrease steadily as the number of iterations increases, and then level off after convergence. The figure marks the early stopping points used in cross-validation (such as around the 65th iteration), at which the validation set loss reaches its minimum without overfitting, effectively ensuring the model's generalization ability.

• 

  Figure 3. Overall classification performance of the machine learning model on internal and external validation sets. The performance metrics include accuracy, macro-average precision, macro-average recall, and macro-average F1 score. On the internal validation set, the accuracy, macro-average precision, macro-average recall, and macro-average F1 score were 0.97, 0.96, 0.95, and 0.96, respectively. On the external validation set, the values were 0.97, 0.97, 0.94, and 0.96, respectively. The highly consistent results between the two independent datasets demonstrate the model's robust generalizability and stable performance in grading anesthesia risk.

• 

  Figure 4. Confusion matrix heatmap of the machine learning model for ASA classification on the internal (a) and external validation set (b). The rows represent the ground truth labels (ASA classification I, II, and III) assessed by anesthesiologists, and the columns represent the predicted labels by the machine learning model. Each cell displays the absolute case count and the corresponding row percentage (recall rate). The diagonal cells (shaded) indicate the proportions of correct classifications, demonstrating high overall accuracy. Notably, all misclassifications are confined to adjacent classes (e.g., Class Ⅰ misclassified as Class Ⅱ, Class Ⅲ misclassified as Class Ⅱ), with no cross-class errors (e.g., Class Ⅲ misclassified as Class Ⅰ). This pattern confirms the model's understanding of the ordinal nature of ASA classification and supports its clinical acceptability.

• 

  Figure 5. Performance of the machine learning model for identifying high-risk patients on internal and external validation sets. ASA classes I and II were defined as low risk, while Class III was defined as high-risk. The model's performance was evaluated using accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Kappa coefficient. On the internal validation set (n = 120), the model achieved an accuracy of 0.98, sensitivity of 0.95, specificity of 0.98, PPV of 0.91, NPV of 0.99, and Kappa of 0.92. On the external validation set (n = 100), the corresponding values were 0.98, 0.93, 0.99, 0.93, 0.99, and 0.92. The consistently high performance across both datasets indicates that the model has good generalizability in distinguishing high-risk from low-risk patients, with excellent negative predictive value (0.99) and substantial to almost perfect agreement (Kappa > 0.90).

• 

  Figure 6. Relative contribution of clinical feature categories to ASA classification prediction based on SHAP analysis. Features were grouped into seven clinical categories according to their physiological systems. The contribution of each category was calculated as the mean absolute SHAP value across all samples, expressed as a percentage of the total. Baseline characteristics and lifestyle habits (30.6%) and cardiovascular system (30.3%) were the two dominant contributors, together accounting for over 60% of the model's predictions.