Background Machine learning (ML) models for suicide risk prediction show promise, but their real-world discriminative ability and the sources of performance variation remain unclear. Methods We systematically searched databases for studies evaluating statistical or ML suicide prediction models in real-world clinical settings. Random-effects meta-analyses pooled the area under the receiver operating characteristic curve (AUC) overall and by outcome type. We conducted exploratory univariable meta-regression with cluster-robust inference for outcome type, evaluation timing, and model type. Sensitivity analyses assessed the influence of within-study dependence. Results Nine studies (20 evaluations) were included. The overall pooled AUC was 0.849 (95% CI 0.827–0.869). Pooled AUCs were 0.865 (0.840–0.887) for suicide attempts, 0.835 (0.825–0.844) for suicidal ideation, and 0.842 (0.797–0.878) for suicide death. Heterogeneity was extreme (I 2 = 99.9%). Sensitivity analyses yielded similar pooled estimates. Exploratory univariable meta-regression showed no clear associations for outcome type, evaluation timing, or model type. Conclusions Suicide prediction models show good real-world discrimination, but performance varies substantially across evaluations. We found no clear evidence that outcome type, evaluation timing, or model type explained this heterogeneity. These findings support rigorous local validation and workflow-sensitive implementation rather than assuming that more complex algorithms will perform better in practice.
Kim et al. (Sun,) studied this question.