Multisite Pragmatic Cluster-Randomized Controlled Trial of the CONCERN Early Warning System

Abstract Importance Late predictions of hospitalized patient deterioration, resulting from early warning systems (EWS) with limited data sources and/or a care team’s lack of shared situational awareness, contribute to delays in clinical interventions. The COmmunicating Narrative Concerns Entered by RNs (CONCERN) Early Warning System (EWS) uses real-time nursing surveillance documentation patterns in its machine learning algorithm to identify patients’ deterioration risk up to 42 hours earlier than other EWSs. Objective To test our a priori hypothesis that patients with care teams informed by the CONCERN EWS intervention have a lower mortality rate and shorter length of stay (LOS) than the patients with teams not informed by CONCERN EWS. Design One-year multisite, pragmatic controlled clinical trial with cluster-randomization of acute and intensive care units to intervention or usual-care groups. Setting Two large U.S. health systems. Participants Adult patients admitted to acute and intensive care units, excluding those on hospice/palliative/comfort care, or with Do Not Resuscitate/Do Not Intubate orders. Intervention The CONCERN EWS intervention calculates patient deterioration risk based on nurses’ concern levels measured by surveillance documentation patterns, and it displays the categorical risk score (low, increased, high) in the electronic health record (EHR) for care team members. Main Outcomes and Measures Primary outcomes: in-hospital mortality, LOS; survival analysis was used. Secondary outcomes: cardiopulmonary arrest, sepsis, unanticipated ICU transfers, 30-day hospital readmission. Results A total of 60 893 hospital encounters (33 024 intervention and 27 869 usual-care) were included. Both groups had similar patient age, race, ethnicity, and illness severity distributions. Patients in the intervention group had a 35.6% decreased risk of death (adjusted hazard ratio HR, 0.644; 95% confidence interval CI, 0.532-0.778; P<.0001), 11.2% decreased LOS (adjusted incidence rate ratio, 0.914; 95% CI, 0.902-0.926; P<.0001), 7.5% decreased risk of sepsis (adjusted HR, 0.925; 95% CI, 0.861-0.993; P=.0317), and 24.9% increased risk of unanticipated ICU transfer (adjusted HR, 1.249; 95% CI, 1.093-1.426; P=.0011) compared with patients in the usual-care group. Conclusions and Relevance A hospital-wide EWS based on nursing surveillance patterns decreased in-hospital mortality, sepsis, and LOS when integrated into the care team’s EHR workflow. Trial Registration ClinicalTrials.gov Identifier: NCT03911687 https://clinicaltrials.gov/ct2/show/NCT03911687 Key Points Question Do patients whose care team receive the CONCERN Early Warning System (EWS) intervention have a lower mortality rate and shorter length of stay than patients in the usual-care group? Findings In this multisite, pragmatic cluster-randomized controlled clinical trial that included 60 893 hospital patient encounters, patients whose care team received the CONCERN EWS intervention had a 35.6% decreased risk of death and 11.2% shorter length of stay compared with those in the usual-care group. Meaning A machine learning-based EWS modeled on nursing surveillance patterns significantly decreased the risk of inpatient deterioration events.