A rule-based NLP system accurately detected bicuspid aortic valve from echocardiographic reports with a precision of 0.925, sensitivity of 0.939, and F1-score of 0.932.
Observational (n=3,478,658)
Yes
Does a rule-based NLP system accurately detect bicuspid aortic valve from echocardiographic reports?
A rule-based NLP system can accurately identify patients with bicuspid aortic valves from millions of unstructured echocardiographic reports, enabling large-scale retrospective cohort creation.
Effect estimate: F1-score 0.932
BACKGROUND: Bicuspid aortic valve (BAV) is the most common congenital heart defect but often evades timely diagnosis due to variable clinical presentations. Prior to October 2024, no specific diagnosis code existed for BAV, limiting retrospective identification. OBJECTIVES: The purpose of this study was to develop and validate a natural language processing (NLP) system for automated extraction of heart valve morphology from echocardiographic reports, with focus on BAV detection. METHODS: We developed a rule-based NLP system using MedSpaCy to analyze echocardiographic reports from the Veterans Affairs Corporate Data Warehouse. The system was trained on 555 manually annotated reports and validated on 170 held-out reports. System performance was evaluated on valve leaflet structure identification. RESULTS: The NLP system achieved excellent performance for BAV detection with a precision of 0.925, a sensitivity of 0.939, and an F1-score of 0.932. When applied to 14,453,591 echocardiographic documents from 3,478,658 patients, the system identified 83,461 patients (2.40%) with affirmed BAV. Among patients identified by the International Classification of Diseases-10 code Q23.81, NLP showed 86.1% concordance, with manual review confirming NLP accuracy in discordant cases. CONCLUSIONS: This NLP approach enables large-scale retrospective identification of BAV patients from clinical text, creating the largest BAV cohort to date and facilitating future cardiovascular research and clinical decision-making.
Bowles et al. (Tue,) conducted a observational in Bicuspid aortic valve (n=3,478,658). Natural language processing (NLP) system vs. ICD-10 code Q23.81 and manual review was evaluated on Valve leaflet structure identification (BAV detection) (F1-score 0.932). A rule-based NLP system accurately detected bicuspid aortic valve from echocardiographic reports with a precision of 0.925, sensitivity of 0.939, and F1-score of 0.932.