Special Issue on CDS Failures: A Measurement Science Framework to Optimize CDS for Opioid Use Disorder Treatment in the ED

Objectives: In the EMergency department-initiated BuprenorphinE for opioid use Disorder (EMBED) trial, a clinical decision support (CDS) tool had no effect on rates of buprenorphine initiation in emergency department (ED) patients with opioid use disorder. The Agency for Healthcare Research and Quality (AHRQ) recently released a CDS Performance Measure Inventory to guide data-driven CDS development and evaluation. Through partner co-design, we tailored AHRQ inventory measures to evaluate EMBED CDS performance and drive improvements. Methods: Relevant AHRQ inventory measures were selected and adapted using a partner co-design approach grounded in consensus methodology, with three iterative, multidisciplinary partner working group sessions involving stakeholders from various roles and institutions; meetings were followed by post-meeting surveys. The co-design process was divided into conceptualization, specification, and evaluation phases building on the Centers for Medicare and Medicaid Services’ measure life cycle framework. Final measures were evaluated in 3 EDs in a single health system from 1/1/2023-12/31/2024. Results: The partner working group included 25 members. During conceptualization, 13 initial candidate metrics were narrowed to 6 priority categories. These were further specified and validated as the following measures, presented with preliminary values based on use of the current (i.e., pre-optimization) EMBED CDS: eligible encounters with CDS engagement, 5.0% (95% CI: 4.3-5.8%); teamwork on ED initiation of buprenorphine, 39.9% (32.5%-47.3%); proportion of eligible users who used EMBED, 58.3% (50.9%-65.8%); time spent on EMBED 29.0 seconds (20.4-37.7 seconds); proportion of buprenorphine orders placed through EMBED, 6.5% (3.4%-9.6%); and task completion, 13.8% (8.9%-18.7%) for buprenorphine order/prescription. Conclusions: A measurement science framework informed by partner co-design was a feasible approach to develop measures to guide CDS improvement. Subsequent research could adapt this approach to evaluate other CDS applications.