Generating actionable insights to support point-of-care suicide risk decision-making in a safety-net healthcare system: a machine learning approach to predicting dynamic risk of intentional self-harm

Background Suicide rates have increased over the last couple of decades globally, particularly in the United States and among populations with lower economic status who present at safety-net healthcare systems. Recently, predictive models for suicide risk have shown promise; however, a model for this specific population does not exist. Objective To develop a predictive risk model of suicide and intentional self-harm (ISH) for patients presenting at the psychiatric emergency department (ED) of JPS Health Network, a safety net medical and mental healthcare system in Texas. Methods The study used structured and unstructured electronic medical record (EMR) data (2015–2019) and local medical examiner data (2015–2020) to create predictors and outcome variables. All psychiatric ED notes during calendar years 2018 and 2019 were reviewed using natural language processing to identify presentations for any level of self-harm and subsequent manual review of identified visits to accurately classify ED presentations for treatment of an act of intentional self-harm meeting study criteria. Data from 15 987 patients were used to develop and validate a machine learning-based predictive model that leverages rolling window methodology to predict risk repeatedly across a patient’s trajectory. Feature engineering played a prominent role in defining new predictors. Findings The best model (XGBoost) achieved the area under the receiver operating characteristic curve of 0.81 for 30-day predictions and demonstrated concentration of ISH and suicide attempt events in high-risk quantiles of risk (65% had events in top 0.1% quantile). The predicted risk can be translated into a propensity of events (80% at the highest predicted risk) to facilitate clinical interpretation. Conclusions Machine learning-based models can be used with standard EMRs to identify patients presenting at the psychiatric ED with a high risk of ISH and suicide attempts within the next 30 days. Clinical implications Integration of a predictive model can significantly aid clinical decision-making in safety-net psychiatric EDs.