Harnessing Natural Language Processing to Identify Documentation of Serious Illness Communication for Patients with Decompensated Cirrhosis

Introduction: Given the high mortality of patients with decompensated cirrhosis (DC), there is increasing focus on improving serious illness communication (SIC) for this population. However, SIC documentation in the electronic health record (EHR) is often unstructured and difficult to find. We aimed to evaluate the ability to use natural language processing (NLP) to identify SIC documentation in clinical notes from patients with DC. Methods: In a single-center cohort of adult patients with DC who were evaluated for liver transplantation between 1/1/2010 and 12/31/2017 and died by 6/30/2018, we developed a semi-automated NLP approach to identify SIC documentation in clinical notes. All inpatient and outpatient notes from one year until three days prior to death were extracted from the EHR. NLP software with semi-automated chart review was applied to identify SIC documentation across four domains: goals of care conversations, code status limitations, specialist palliative care involvement, and hospice assessment. The performance of NLP was compared to gold standard manual chart review. Results: 196 unique patients with 14,062 notes were included in the study. In the gold standard data set, NLP achieved F1 scores ranging from 0.91-1.0 across all 4 SIC domains. Identification of SIC documentation required 6.8 minutes per patient using NLP, compared to 41.5 minutes per patient using manual chart review. 48% of patients had no SIC documentation. Discussion: NLP is more efficient and as accurate as manual chart review for identifying SIC documentation in the EHR for patients with DC and can be used at scale for quality improvement initiatives and clinical trials.