What is the clinical evidence from this study?

Study design: Observational. Population: Atrial fibrillation (n=62106). Intervention: EHR-based ISTH bleeding algorithms vs. Manual chart review. Primary outcome: Sensitivity of ISTH major bleeding algorithm (Sensitivity 0.91, 95% CI 0.86, 0.94).

May 1, 2026Open Access

Validation of electronic health record-based International Society on Thrombosis and Haemostasis bleeding algorithms in a US health system

Key Result

The EHR-based ISTH major bleeding algorithm demonstrated high sensitivity of 0.91 and specificity of 0.92, whereas the clinically relevant nonmajor bleeding algorithm showed suboptimal sensitivity of 0.66.

Study Design

Type

Observational (n=62,106)

Multicenter

Yes

Structured PICO

Do EHR-based algorithms accurately identify ISTH-defined major and clinically relevant nonmajor bleeding in adults with atrial fibrillation on oral anticoagulants?

Population

62,106 adults (mean age 72.6 years, 42.9% female) with atrial fibrillation and prior oral anticoagulant use, of which 433 underwent chart review to validate EHR bleeding algorithms over a median follow-up of 4.3 years.

Exposure

EHR-based algorithms to identify ISTH-defined major and clinically relevant nonmajor (CRNM) bleeding

Comparator

Clinician chart review (reference standard) in 433 conditionally sampled patients

Outcome

Sensitivity, specificity, positive predictive values, and negative predictive values for identifying ISTH-defined major and CRNM bleeding

EHR-based algorithms can accurately identify ISTH-defined major bleeding events in real-world atrial fibrillation patients, though sensitivity for clinically relevant nonmajor bleeding is suboptimal.

Main Result

Effect estimate: Sensitivity 0.91 (95% CI 0.86, 0.94)

Limitations

EHR-based algorithms defined fatal death as death occurring within 45 days postencounter, which may result in increased false positives
Algorithms used procedural codes to identify blood transfusions, but details on the number of units transfused and the timing of transfusion(s) were not captured
Unable to apply hemoglobin drop bleed verification requirements for encounters outside the health system
Performance metrics were calculated directly from adjudicated 2x2 tables without applying weights to account for conditional sampling
Validation study was conducted only at a single integrated health system

Abstract

Background: Novel algorithms were developed to identify International Society on Thrombosis and Haemostasis (ISTH)-defined major and clinically relevant nonmajor (CRNM) bleeding events using electronic health record (EHR) data. Objectives: We applied and validated these EHR-based algorithms in a large US integrated health system. Methods: We conducted a retrospective cohort and chart review study of adults with atrial fibrillation and ≥1 oral anticoagulant (OAC) prescription between 2013 and 2021. EHR-based algorithms were applied to identify ISTH-defined major and CRNM bleeding from the first OAC fill until disenrollment, death, or June 2022, whichever occurred first. Through conditional sampling, 433 patients were selected for clinician chart review. Sensitivity, specificity, positive predictive values, and negative predictive values were calculated. Results: Among 62,106 adults (mean age, 72.6 years; 57.1% male), the ISTH major and CRNM bleeding event rates per the algorithms were 3.54 and 19.67 per 100 person-years, respectively, during a median follow-up of 4.3 years. Sensitivity and specificity were 0.91 (95% CI, 0.86, 0.94) and 0.92 (95% CI, 0.88, 0.95) for major bleeding, respectively, and 0.66 (95% CI, 0.54, 0.76) and 0.86 (95% CI, 0.82, 0.90) for CRNM bleeding, respectively. The positive and negative predictive values were 0.91 (95% CI, 0.87, 0.94) and 0.92 (95% CI, 0.89, 0.95) for major bleeding, respectively, and 0.52 (95% CI, 0.44, 0.60) and 0.92 (95% CI, 0.90, 0.94) for CRNM bleeding, respectively. Conclusion: The EHR-based ISTH major bleeding algorithm demonstrated high sensitivity and specificity, while the CRNM bleeding algorithm showed suboptimal sensitivity. The EHR-based ISTH algorithms may enhance understanding of OAC safety using real-world evidence.

Bookmark

View Full Paper