The CHIEF natural language processing system classified heart failure hospitalizations for the CHI19 quality measure with 98.9% sensitivity and 98.7% positive predictive value compared to a reference standard.
Observational (n=1,083)
Sí
Does the CHIEF NLP system accurately automate the extraction of heart failure quality measures compared to human review in VA inpatients?
An automated NLP system accurately extracted heart failure quality measures from electronic health records, potentially improving the efficiency of quality reporting.
BACKGROUND: We developed an accurate, stakeholder-informed, automated, natural language processing (NLP) system to measure the quality of heart failure (HF) inpatient care, and explored the potential for adoption of this system within an integrated health care system. OBJECTIVE: To accurately automate a United States Department of Veterans Affairs (VA) quality measure for inpatients with HF. METHODS: We automated the HF quality measure Congestive Heart Failure Inpatient Measure 19 (CHI19) that identifies whether a given patient has left ventricular ejection fraction (LVEF) <40%, and if so, whether an angiotensin-converting enzyme inhibitor or angiotensin-receptor blocker was prescribed at discharge if there were no contraindications. We used documents from 1083 unique inpatients from eight VA medical centers to develop a reference standard (RS) to train (n=314) and test (n=769) the Congestive Heart Failure Information Extraction Framework (CHIEF). We also conducted semi-structured interviews (n=15) for stakeholder feedback on implementation of the CHIEF. RESULTS: The CHIEF classified each hospitalization in the test set with a sensitivity (SN) of 98.9% and positive predictive value of 98.7%, compared with an RS and SN of 98.5% for available External Peer Review Program assessments. Of the 1083 patients available for the NLP system, the CHIEF evaluated and classified 100% of cases. Stakeholders identified potential implementation facilitators and clinical uses of the CHIEF. CONCLUSIONS: The CHIEF provided complete data for all patients in the cohort and could potentially improve the efficiency, timeliness, and utility of HF quality measurements.
Garvin et al. (Mon,) conducted a observational in Heart failure (n=1,083). Congestive Heart Failure Information Extraction Framework (CHIEF) NLP system vs. Reference Standard (human annotation) and External Peer Review Program (EPRP) was evaluated on Sensitivity for patient-level classification of meeting the CHI19 measure (95% CI 97.8-99.5). The CHIEF natural language processing system classified heart failure hospitalizations for the CHI19 quality measure with 98.9% sensitivity and 98.7% positive predictive value compared to a reference standard.