An augmented intelligence-enabled workflow using machine learning identified stage D heart failure with a positive predictive value of 60% and sensitivity of 25% in a holdout test set.
Observational (n=14,846)
Does an augmented intelligence-enabled workflow using machine learning improve the identification and referral of patients with advanced (stage D) heart failure?
An EHR-integrated machine learning model can assist in identifying patients with advanced heart failure to facilitate timely specialist referral, though it demonstrated low sensitivity.
BACKGROUND: Timely referral for specialist evaluation in patients with advanced heart failure (HF) is a Class 1 recommendation. However, the transition from stage C HF to advanced or stage D HF often goes undetected in routine care, resulting in delayed referral and higher mortality rates. OBJECTIVES: The authors sought to develop an augmented intelligence-enabled workflow using machine learning to identify patients with stage D HF and streamline referral. METHODS: We extracted data on HF patients with encounters from January 1, 2007, to November 30, 2020, from a HF registry within a regional, integrated health system. We created an ensemble machine learning model to predict stage C or stage D HF and integrated the results within the electronic health record. RESULTS: In a retrospective data set of 14,846 patients, the model had a good positive predictive value (60%) and low sensitivity (25%) for identifying stage D HF in a 100-person, physician-reviewed, holdout test set. During prospective implementation of the workflow from April 1, 2021, to February 15, 2022, 416 patients were reviewed by a clinical coordinator, with agreement between the model and the coordinator in 50.3% of stage D predictions. Twenty-four patients have been scheduled for evaluation in a HF clinic, 4 patients started an evaluation for advanced therapies, and 1 patient received a left ventricular assist device. CONCLUSIONS: An augmented intelligence-enabled workflow was integrated into clinical operations to identify patients with advanced HF. Endeavors such as this require a multidisciplinary team with experience in design thinking, informatics, quality improvement, operations, and health information technology, as well as dedicated resources to monitor and improve performance over time.
Cheema et al. (Sat,) conducted a observational in Heart failure (n=14,846). Augmented intelligence-enabled workflow using machine learning was evaluated on Identification of stage D heart failure. An augmented intelligence-enabled workflow using machine learning identified stage D heart failure with a positive predictive value of 60% and sensitivity of 25% in a holdout test set.