Automated Identification of Heart Failure with Reduced Ejection Fraction using Deep Learning-based Natural Language Processing

Figure 1.. Model Development. Hospitalizations for heart failure at Yale New Haven Hospital with an antecedent echocardiography were included for model development. The schematic represents data processing, model development, and evaluation of model performance on validation sets, including Northwestern Medicine, community hospitals of Yale New Haven health system, and MIMIC-III database. Abbreviations: YNHH, Yale New Haven Hospital; HFrEF, heart failure with reduced ejection fraction; MIMIC, Medical Information Mart for Intensive Care.

Figure 2.. Model Performance. The graphs represent receiver operating characteristic (left) and precision-recall (right) curves in identifying HFrEF on clinical notes from the held-out test set at Yale New Haven Hospital. Abbreviations: ROC, receiver operating characteristic curve; PRC, precision recall curve; AUC, area under curve.

Figure 3.. External Validation. Performance of the model in identifying HFrEF was evaluated on discharge summaries from three external sets, including 1) Northwestern Medicine, 2) Community hospital of Yale New Haven health system, and 3) MIMIC-III. The figure represents receiver operating characteristic (left) and precision-recall (right) curves for these analyses. Abbreviations: YNH, Yale New Haven; MIMIC, Medical Information Mart for Intensive Care III; AUC, area under curve.

Figure 4.. External Validation. The bar graph represents model performance in detecting HFrEF from discharge summaries of hospitalizations with heart failure at Northwestern Medicine (blue), community hospitals of Yale New Haven health (pink) and MIMIC-III (yellow). Abbreviations: AUROC, area under receiving operating characteristics; AUPRC, area under precision recall curve; PPV, positive predictive value; NPV, negative predictive value; MIMIC, Medical Information Mart for Intensive Care III.

Figure 5.. Disease Classification and Use of Guideline Directed Therapies. (A) Nested pie chart demonstrates disease classification based on echocardiography measurements of LVEF (true label, outer circle), model predictions of disease phenotype (middle circle), and chart-documented diagnosis codes (inner circle). Blue represents a diagnosis of HFrEF and red represents other phenotypes of heart failure; (B) Use of guideline-directed therapies among individuals with LVEF < 40% who were correctly reclassified from a chart-diagnosis of non-HFrEF to a model-based diagnosis of HFrEF. Abbreviations: ARNI, angiotensin receptor-neprilysin inhibitor; ARB, angiotensin receptor blocker; ACEI, angiotensin converting enzyme inhibitor; SGLT2i, sodium glucose cotransporter 2 inhibitor; MRA, mineralocorticoid receptor antagonist.

Figure 6.. Predictive Keywords for Heart failure Phenotypes. Figure represents Local Interpretable Model-agnostic Explanations (LIME) for model predictions of HFrEF from discharge summaries. (A) The most predictive keywords for HFrEF with corresponding coefficients based on the LIME analysis of the top 100 notes with the most confident model predictions. (B) A real-world example of LIME analysis on a discharge summary representing local dependencies. The color intensity of each highlighted keyword represents its contributions to the model-predicted probability of a HFrEF (orange) vs a non-HFrEF phenotype (blue). The deidentified discharge summary describes the hospitalization of an individual after initial presentation with heart failure and left ventricular ejection fraction of 30% on echocardiography. The model predicted a 99% probability for a HFrEF from this discharge summary. Abbreviations: MRI, magnetic resonance imaging; ICM, ischemic cardiomyopathy; VAD, ventricular assist device; NICM, non-ischemic cardiomyopathy; PAD, peripheral arterial disease; HFrEF, heart failure with reduced ejection fraction; PMH, past medical history; USH, usual state of health; ED, emergency department;…