Does increasing model complexity improve HEART-based chest pain risk stratification? A multicentre development and held-out emergency department validation study

Q: What are the key findings of this study?

Data-driven modeling (CVLR and XGBoost) improved precision-recall performance (AUPRC 0.237 and 0.244 vs 0.187, p<0.001) but did not significantly improve AUROC or expand low-risk classification compared to the fixed HEART score.

Q: What is the clinical evidence from this study?

Study design: Cohort. Population: Chest pain with suspected acute coronary syndrome (n=47853). Intervention: Data-driven reweighting of HEART components (CVLR and XGBoost) vs. Conventional fixed-point HEART score. Primary outcome: 30-day Major Adverse Cardiac Events (MACE) (AUROC difference 0.002 (CVLR vs HEART), 95% CI -0.005-0.008, p=0.604).

Q: What does this research mean for the field?

Data-driven reweighting of HEART score components improves precision-recall performance for 30-day MACE prediction, but increasing model complexity to XGBoost does not improve discrimination or expand the low-risk population compared to logistic regression or the fixed score. Novelty: ClaimNovelty.CONFIRMATORY. Consensus alignment: ConsensusAlignment.NEUTRAL.

Hao Wang; Jenna R. Williamson; Hunter Scarborough; Devin Sandlin

doi:10.1136/openhrt-2026-004391

July 1, 2026Open HeartOpen Access

Does increasing model complexity improve HEART-based chest pain risk stratification? A multicentre development and held-out emergency department validation study

View Full Paper

Ask AI

Bookmark

Share

Key result

Machine learning models fail to improve discrimination or risk classification over the standard HEART score.

AUROC difference 0.002 (CVLR vs HEART)
95% CI -0.005-0.008
P=0.604
n=47,853

Why the study?

The fixed-point structure of the HEART score may not optimally represent relationships between its components and major adverse cardiac events, motivating evaluation of more flexible modelling approaches.

Does data-driven statistical modeling or machine learning (XGBoost) of the HEART score components improve 30-day MACE prediction in adult emergency department patients with chest pain compared to the conventional fixed-point HEART score?

Key Points

To determine whether data-driven and non-linear modelling approaches improve 30-day major adverse cardiac event (MACE) prediction compared to the conventional fixed-point HEART score when using the same five clinical components.
Retrospective multicentre cohort study of 47 853 adult emergency department patients presenting with chest pain across six emergency departments between 2021 and 2023.
Data from five emergency departments (n=39 587) were used for model development, and a geographically held-out sixth emergency department (n=8266) was used for validation.
The conventional HEART score was evaluated against categorical cross-validated logistic regression (CVLR) and extreme gradient boosting (XGBoost) for predicting 30-day MACE (acute myocardial infarction, coronary revascularisation, or all-cause mortality).
In the held-out validation cohort (30-day MACE rate of 3.23%), discrimination was similar across all three models: AUROC was 0.855 for HEART, 0.857 for CVLR, and 0.856 for XGBoost (pairwise p>0.05).
At comparable-sensitivity operating points, low-risk classifications were 63.74% for HEART, 59.39% for CVLR, and 61.58% for XGBoost, with observed low-risk MACE rates of 0.53%, 0.51%, and 0.57%, respectively.

Study Design

Type

Cohort (n=47,853)

Multicenter

Yes

Structured PICO

Does data-driven statistical modeling or machine learning (XGBoost) of the HEART score components improve 30-day MACE prediction in adult emergency department patients with chest pain compared to the conventional fixed-point HEART score?

P

Population

47,853 adult emergency department patients presenting with chest pain and a documented HEART score across 6 sites, evaluated for 30-day major adverse cardiac events.

E

Exposure

Data-driven prediction models (categorical cross-validated logistic regression [CVLR] and extreme gradient boosting [XGBoost]) utilizing only the five documented HEART score components.

C

Comparator

Conventional fixed-point HEART score (History, ECG, Age, Risk factors, Troponin).

O

Outcome

30-day major adverse cardiac events (MACE), a composite of acute myocardial infarction (AMI), coronary revascularization (PCI or CABG), or all-cause mortality, identified via EHR and administrative coding.composite

Main Result

Effect estimate: AUROC difference 0.002 (CVLR vs HEART) (95% CI -0.005-0.008)

p-value: p=0.604

When restricted to the same five clinical predictors, increasingly complex machine learning algorithms do not significantly improve overall discrimination or safely expand low-risk classification compared to the conventional HEART score for emergency department chest pain evaluation.

Limitations

Retrospective observational design is subject to selection bias and misclassification
Inclusion was limited to patients with documented HEART scores
The geographically held-out validation ED remained within the same health system, limiting generalizability
EHR-based and administrative outcome ascertainment may miss events outside the available data networks
Restricting the models to only the five HEART components constrained potential predictive performance
Excluded patients transferred directly, undergoing emergent cardiac catheterization, or dying in the ED
Retrospective observational design subject to selection bias and misclassification
Subjective nature of the History component
Exclusion of patients directly transferred, taken emergently to catheterization, or who died in the ED
Validation was geographically held-out but remained within the same health system
Composite outcome components differ in mechanism and importance, and out-of-network events could be missed
Models were intentionally restricted to only the five HEART components, constraining overall predictive performance

Implication

Machine learning models do not enhance discrimination or low-risk classification beyond the HEART score; leaves open whether AUPRC gains justify added complexity in future validation.

Abstract

BACKGROUND AND OBJECTIVES: The History, ECG, Age, Risk factors and Troponin (HEART) score is widely used for emergency department (ED) chest pain risk stratification, but its fixed-point structure may not optimally represent relationships between its components and major adverse cardiac events (MACE). We evaluated whether increasingly flexible modelling approaches improve prediction when restricted to the same five HEART components. METHODS: We conducted a retrospective multicentre cohort study of 47 853 adult ED patients with chest pain and documented HEART scores across six EDs from 2021 to 2023. Data from five EDs (n=39 587) were used for model development and one geographically held-out ED (n=8266) was used for validation. The conventional HEART score was compared with categorical cross-validated logistic regression (CVLR) and extreme gradient boosting (XGBoost) for predicting 30-day MACE. Performance was assessed using the area under the receiver operating characteristic (AUROC), the area under the precision-recall curve (AUPRC), the Brier score, paired bootstrap comparisons and clinical performance at comparable-sensitivity operating points. RESULTS: In the held-out validation cohort, 267 patients (3.23%) experienced 30-day MACE. AUROCs were similar for HEART (0.855), CVLR (0.857) and XGBoost (0.856; pairwise p>0.05). In contrast, both data-driven models demonstrated significantly higher AUPRC than HEART: CVLR (0.237 vs 0.187; Δ=0.050, p<0.001) and XGBoost (0.244 vs 0.187; Δ=0.057, p<0.001) with no significant difference between CVLR and XGBoost (p=0.305). At comparable-sensitivity operating points, HEART, CVLR and XGBoost classified 63.74%, 59.39% and 61.58% of patients as low risk, respectively, with corresponding low-risk MACE rates of 0.53%, 0.51% and 0.57%. CONCLUSIONS: Data-driven reweighting of HEART components improved precision-recall performance and risk ranking compared with fixed HEART scoring. However, XGBoost provided no incremental benefit over categorical CVLR, and neither approach expanded low-risk classification at comparable sensitivity.

Discussion

No takes yet. Share an insight, caveat, or question.

Cite This Study

Wang et al. (2026) conducted a cohort in Chest pain with suspected acute coronary syndrome (n=47,853). Data-driven reweighting of HEART components (CVLR and XGBoost) vs. Conventional fixed-point HEART score was evaluated on 30-day Major Adverse Cardiac Events (MACE) (AUROC difference 0.002 (CVLR vs HEART), 95% CI -0.005-0.008, p=0.604). Data-driven modeling (CVLR and XGBoost) improved precision-recall performance (AUPRC 0.237 and 0.244 vs 0.187, p<0.001) but did not significantly improve AUROC or expand low-risk classification compared to the fixed HEART score.

synapsesocial.com/papers/6ad0fcf33afa093ea9ad0102 https://doi.org/10.1136/openhrt-2026-004391

View Full Paper

Ask AI

Bookmark

Share

Does increasing model complexity improve HEART-based chest pain risk stratification? A multicentre development and held-out emergency department validation study

Key Points

Study Design

Structured PICO

Main Result

Limitations

Implication

Abstract

Discussion

Cite This Study

Does increasing model complexity improve HEART-based chest pain risk stratification? A multicentre development and held-out emergency department validation study

Key Points

Study Design

Structured PICO

Main Result

Limitations

Implication

Abstract

Discussion

Cite This Study

Keep reading

Keep reading