A generalizable deep learning model for predicting opioid use disorder using language modeling for cross-EHR data harmonization

The opioid use disorder (OUD) epidemic in the United States has been a major public health crisis. We developed OUD-REY ( OUD R isk E valuator at 1 Y ear) - a deep learning model that uses longitudinal electronic health record (EHR) data from the All of Us research program to predict risk of OUD at one year for patients with an opioid prescription. On internal validation, OUD-REY achieved an AUROC of 91.2% and AUPRC of 77.0%. We leveraged a pretrained language model to accommodate thousands of unseen, out-of-vocabulary clinical features in a patient’s longitudinal EHR history. We externally validated our model on a new dataset released by All of Us that was temporally separated from our development set and on a dataset containing ~6 million patients from medical centers of the Veterans Affairs (VA) nationwide over a span of 20 years. On external validation, our model had an AUROC and AUPRC of 86.4% and 23.1% (OUD prevalence of 2.9%), respectively, on our temporally separated All of Us validation set. It had an AUROC and AUPRC of 85.5% and 19.5% (OUD prevalence of 3.5%), respectively, on the VA dataset. Our fairness auditing showed that the model was generally fair for patients with different genders, race, and ethnic backgrounds. This model may potentially improve prevention and care of OUD.