Development and external validation of a machine-learning model for preoperative prediction of impacted ureteral stones in patients undergoing ureteroscopy: a retrospective two-centre study

Background Preoperative identification of impacted ureteral stones may inform procedural planning. This study aimed to develop and externally validate a prediction model using clinical and non-contrast computed tomography (CT) features. Methods This retrospective two-centre study used routinely collected clinical, laboratory, preoperative non-contrast CT and operative data from the First Affiliated Hospital of Jinzhou Medical University and Huludao Central Hospital. It included consecutive adults aged 18–80 years who underwent ureteroscopic treatment and had complete clinical, imaging and operative outcome information. The outcome was intraoperative surgeon-reported impaction according to a predefined operative definition. Nine algorithms were evaluated using five-fold outer cross-validation repeated five times, with predictor selection, preprocessing, tuning and algorithm selection restricted to the relevant training partitions. The final algorithm was selected by training-stage selection frequency, refitted on the full development cohort and applied externally without retraining or recalibration. Discrimination, calibration, Brier scores and classification performance at a fixed probability threshold of 0.50 were assessed. Shapley Additive exPlanations (SHAP) was used for post-hoc model attribution. Results Impaction occurred in 209 development patients (38.3%) and 38 external-validation patients (37.6%). Random forest (RF) was selected in 8 of 25 outer-training resamples and retained six predictors: flank pain, stone attenuation, proximal ureteral width, ureteral wall thickness(UWT), stone length, and hydronephrosis area. Its patient-level nested out-of-fold area under the receiver operating characteristic curve (AUC) was 0.917 (95% confidence interval(CI), 0.893–0.941), compared with 0.909 (95% CI, 0.883–0.935) for logistic regression. Internal and external Brier scores were 0.111 (95% CI, 0.097–0.126) and 0.081 (95% CI, 0.057–0.109), respectively. In external validation, the AUC was 0.970 (95% CI, 0.941–0.999), with sensitivity of 0.921 (95% CI, 0.786–0.983) and specificity of 0.873 (95% CI, 0.765–0.944). Calibration slopes were 1.361 internally and 1.982 externally. Ureteral wall thickness had the largest mean absolute SHAP attribution. At the fixed 0.50 threshold, external classification produced eight false positives and three false negatives. Conclusions The six-predictor random forest model demonstrated promising discrimination for preoperative identification of impacted ureteral stones, with supporting evidence from independent external validation. Using routinely available clinical and CT features, the model may facilitate preoperative risk stratification. Further multicenter validation, calibration assessment, and evaluation of clinical utility are needed before clinical implementation.