June 1, 2021BMJ OpenOpen Access

Ascertaining Framingham heart failure phenotype from inpatient electronic health record data using natural language processing: a multicentre Atherosclerosis Risk in Communities (ARIC) validation study

View Full Paper

Ask AI

Bookmark

Why the study?

To determine the performance of natural language processing using free-text clinical notes and reports from hospitalised patients to ascertain Framingham heart failure criteria and phenotype.

Does natural language processing accurately ascertain Framingham heart failure phenotype compared to manual chart review in hospitalized patients?

Structured PICO

Does natural language processing accurately ascertain Framingham heart failure phenotype compared to manual chart review in hospitalized patients?

Population

800 hospitalisations (394 derivation dataset, 406 validation dataset) identified using ICD-9 diagnostic codes indicative of a heart failure event occurring during 2015 from four ARIC study hospitals in the USA.

Intervention

Natural language processing (NLP) on free-text clinical notes and reports to ascertain Framingham heart failure criteria and phenotype

Comparator

Manual medical record review by trained ARIC abstractors (reference standard)

Outcome

NLP performance as measured by sensitivity, specificity, positive-predictive value (PPV) and agreement in ascertainment of Framingham heart failure criteria and phenotype

Natural language processing can accurately ascertain the Framingham heart failure phenotype from free-text electronic health records, potentially improving the efficiency of large-scale epidemiologic surveillance.

Implication

NLP may facilitate efficient HF phenotyping in EHRs; leaves open need for external validation before routine use.

Abstract

OBJECTIVES: Using free-text clinical notes and reports from hospitalised patients, determine the performance of natural language processing (NLP) ascertainment of Framingham heart failure (HF) criteria and phenotype. STUDY DESIGN: A retrospective observational study design of patients hospitalised in 2015 from four hospitals participating in the Atherosclerosis Risk in Communities (ARIC) study was used to determine NLP performance in the ascertainment of Framingham HF criteria and phenotype. SETTING: Four ARIC study hospitals, each representing an ARIC study region in the USA. PARTICIPANTS: A stratified random sample of hospitalisations identified using a broad range of International Classification of Disease, ninth revision, diagnostic codes indicative of an HF event and occurring during 2015 was drawn for this study. A randomly selected set of 394 hospitalisations was used as the derivation dataset and 406 hospitalisations was used as the validation dataset. INTERVENTION: Use of NLP on free-text clinical notes and reports to ascertain Framingham HF criteria and phenotype. PRIMARY AND SECONDARY OUTCOME MEASURES: NLP performance as measured by sensitivity, specificity, positive-predictive value (PPV) and agreement in ascertainment of Framingham HF criteria and phenotype. Manual medical record review by trained ARIC abstractors was used as the reference standard. RESULTS: Overall, performance of NLP ascertainment of Framingham HF phenotype in the validation dataset was good, with 78.8%, 81.7%, 84.4% and 80.0% for sensitivity, specificity, PPV and agreement, respectively. CONCLUSIONS: By decreasing the need for manual chart review, our results on the use of NLP to ascertain Framingham HF phenotype from free-text electronic health record data suggest that validated NLP technology holds the potential for significantly improving the feasibility and efficiency of conducting large-scale epidemiologic surveillance of HF prevalence and incidence.