Long-term Effect of Machine Learning–Triggered Behavioral Nudges on Serious Illness Conversations and End-of-Life Outcomes Among Patients With Cancer

Importance: Serious illness conversations (SICs) between oncology clinicians and patients are associated with improved quality of life and may reduce aggressive end-of-life care. However, most patients with cancer die without a documented SIC. Objective: To test the impact of behavioral nudges to clinicians to prompt SICs on the SIC rate and end-of-life outcomes among patients at high risk of death within 180 days (high-risk patients) as identified by a machine learning algorithm. Design, Setting, and Participants: This prespecified 40-week analysis of a stepped-wedge randomized clinical trial conducted between June 17, 2019, and April 20, 2020 (including 16 weeks of intervention rollout and 24 weeks of follow-up), included 20 506 patients with cancer representing 41 021 encounters at 9 tertiary or community-based medical oncology clinics in a large academic health system. The current analyses were conducted from June 1, 2021, to May 31, 2022. Intervention: High-risk patients were identified using a validated electronic health record machine learning algorithm to predict 6-month mortality. The intervention consisted of (1) weekly emails to clinicians comparing their SIC rates for all patients against peers' rates, (2) weekly lists of high-risk patients, and (3) opt-out text messages to prompt SICs before encounters with high-risk patients. Main Outcomes and Measures: The primary outcome was SIC rates for all and high-risk patient encounters; secondary end-of-life outcomes among decedents included inpatient death, hospice enrollment and length of stay, and intensive care unit admission and systemic therapy close to death. Intention-to-treat analyses were adjusted for clinic and wedge fixed effects and clustered at the oncologist level. Results: The study included 20 506 patients (mean SD age, 60.0 14.0 years) and 41 021 patient encounters: 22 259 (54%) encounters with female patients, 28 907 (70.5%) with non-Hispanic White patients, and 5520 (13.5%) with high-risk patients; 1417 patients (6.9%) died by the end of follow-up. There were no meaningful differences in demographic characteristics in the control and intervention periods. Among high-risk patient encounters, the unadjusted SIC rates were 3.4% (59 of 1754 encounters) in the control period and 13.5% (510 of 3765 encounters) in the intervention period. In adjusted analyses, the intervention was associated with increased SICs for all patients (adjusted odds ratio, 2.09 95% CI, 1.53-2.87; P < .001) and decreased end-of-life systemic therapy (7.5% 72 of 957 patients vs 10.4% 24 of 231 patients; adjusted odds ratio, 0.25 95% CI, 0.11-0.57; P = .001) relative to controls, but there was no effect on hospice enrollment or length of stay, inpatient death, or end-of-life ICU use. Conclusions and Relevance: In this randomized clinical trial, a machine learning-based behavioral intervention and behavioral nudges to clinicans led to an increase in SICs and reduction in end-of-life systemic therapy but no changes in other end-of-life outcomes among outpatients with cancer. These results suggest that machine learning and behavioral nudges can lead to long-lasting improvements in cancer care delivery. Trial Registration: ClinicalTrials.gov Identifier: NCT03984773.