Development of machine learning algorithms for identifying repeat suicide attempters among emergency department patients

Suicide is a major public health concern, particularly in South Korea, and the emergency department provides an important opportunity to identify patients at increased risk. A history of prior suicide attempts is among the strongest predictors of subsequent suicidal behaviour, but it may not always be readily apparent at presentation. This study aimed to develop machine-learning models to identify patients with a history of prior suicide attempts among adults presenting to the emergency department after a suicide attempt, using routinely available demographic and clinical variables. We retrospectively analysed 1,368 adults aged 18 years or older who presented to the emergency department of Hanyang University Hospital after a suicide attempt between January 2021 and June 2023. Eleven demographic and suicide-related variables were used to develop eight models: logistic regression (LR), linear support vector machine (Linear-SVM), decision tree (DT), random forest (RF), gradient boosting machine (GBM), neural network (NN), eXtreme gradient boosting (XGBoost), and naive Bayes (NB). Model performance was assessed using repeated stratified 10-fold cross-validation. Discrimination, calibration, and classification performance were evaluated using the area under the receiver operating characteristic curve (AUC), Brier score, and additional performance measures with 95% confidence intervals. Of the 1,368 patients, 589 (43.1%) had a history of at least one prior suicide attempt. Psychiatric history and age were consistently among the most influential predictors across models. Discrimination was broadly similar for most models, with AUCs ranging from 0.71 to 0.73 and substantial overlap in the 95% confidence intervals. The DT showed lower discrimination, with an AUC of 0.66. Calibration was acceptable for most models (Brier scores approximately 0.21) but poorer for the decision tree (0.27). Models based on routinely available emergency department variables showed moderate ability to identify patients with a history of prior suicide attempts. Psychiatric history and age contributed most consistently to prediction. More complex algorithms did not show a clear performance advantage over simpler models in this limited-variable dataset. Given the single-centre design and absence of external validation, these findings should be considered exploratory.