Miscarriage Risk Prediction for Women With Polycystic Ovary Syndrome Who Achieve Clinical Pregnancy After Fresh Embryo Transfer: Multicenter Development, Geographic External Validation, and Prospective Temporal Validation Study

BACKGROUND Women with polycystic ovary syndrome (PCOS) who achieve clinical pregnancy following fresh in vitro fertilization (IVF)/intracytoplasmic sperm injection (ICSI) carry an elevated miscarriage risk. However, individualized risk stratification after clinical pregnancy confirmation but before outcome determination remains challenging in clinical practice. OBJECTIVE This study aimed to develop, externally and prospectively validate over time, and interpret a machine learning–based risk stratification model for women with PCOS who had achieved a confirmed clinical pregnancy after fresh IVF/ICSI. METHODS This multicenter study included 8911 clinical pregnancy IVF/ICSI cycles across development (n=7241; miscarriages: n=973), external validation (n=1170; miscarriages: n=146), and prospective temporal validation (n=500; miscarriages: n=62) cohorts. Prespecified predictors contained no observed missing values; training-fold median imputation was kept only as a leakage-safe fallback. A total of 13 candidate machine learning and traditional statistical algorithms were compared internally using repeated stratified 5-fold cross-validation (5 repeats) within the development cohort; preprocessing and hyperparameter tuning were performed exclusively in training folds. A prelocked Extreme Gradient Boosting (XGBoost) model was carried forward unchanged for external and prospective temporal validation. Cross-fitted logistic recalibration of averaged XGBoost out-of-fold probabilities was evaluated as a secondary development-sample diagnostic analysis. Model performance encompassed discrimination, precision-recall, calibration, threshold-based metrics, decision curve analysis, risk stratification, and Shapley additive explanations–based interpretability. RESULTS Miscarriage rates were 13.4% (973/7241), 12.5% (146/1170), and 12.4% (62/500) for the development, external validation, and prospective temporal validation cohorts, respectively. In repeated cross-validation, XGBoost (area under the receiver operating characteristic curve [AUC] 0.885, 95% CI 0.874-0.896) and LightGBM (AUC 0.882, 95% CI 0.871-0.893) showed comparable discrimination; the paired XGBoost-LightGBM DeLong test was not significant (Holm-adjusted P=.07). Raw XGBoost out-of-fold probabilities overestimated absolute risk (mean predicted risk 30.3% vs observed risk 13.4%); cross-fitted recalibration within the development cohort improved calibration but was not propagated to validation cohorts. The locked XGBoost model achieved AUCs of 0.787 (95% CI 0.748-0.827; Brier score 0.099) in external validation and 0.763 (95% CI 0.709-0.816; Brier score 0.104) in prospective temporal validation, showing cross-cohort performance attenuation. Risk-quartile stratification using fixed development-cohort cutoffs showed consistent event enrichment across validation cohorts, but positive predictive values remained modest. Shapley additive explanations analysis identified anti-Müllerian hormone, endometrial thickness on the day of human chorionic gonadotropin administration, and basal luteinizing hormone as leading predictors. CONCLUSIONS The locked XGBoost model using routine preoutcome variables may support miscarriage risk stratification and counseling for women with PCOS with clinical pregnancy after fresh IVF/ICSI. Given cross-cohort discrimination loss and calibration drift, it should serve only as a risk-ranking tool, not for diagnosis or treatment selection. Local calibration monitoring is mandatory before any real-world clinical deployment.