Introduction: Benchmarking sepsis outcomes is essential to improving care. CDC and CMS are developing an electronic mortality outcome measure for adult community-onset (CO) sepsis based on CDC’s Adult Sepsis Event (ASE) definition (updated to enhance performance) and applied using electronic health record (EHR) and administrative claims data. We developed a risk-adjustment model for CO-sepsis mortality and assessed its discrimination, calibration, reliability, and validity using 3 large multi-hospital datasets. Methods: We identified CO-sepsis cases by applying ASE criteria to EHR data from 268 hospitals during 2022-2023: HCA Healthcare (n=126 hospitals, MediTech), Mass General Brigham (n=9, Epic), and PINC AI (n=133, various EHRs). We developed a logistic regression model to predict mortality (hospital death or hospice discharge) using administrative and clinical data available within 6-24h of arrival. We assessed discrimination (AUROC), calibration (Brier score), and validated the model using bootstrap methods. We evaluated hospital-level reliability (signal-to-noise ratio) and validity by correlating hospital standardized mortality ratios (SMRs) with CMS quality metrics (SEP-1, 30-day pneumonia mortality, and hospital quality star ratings), as well as expert/patient panel feedback. Results: Of 434,233 CO-sepsis hospitalizations at 268 hospitals, 85,978 (19.8%) ended in death or hospice. The model showed excellent discrimination (AUROC 0.832) and good calibration (Brier score 0.12). Median SMR was 0.99 (10th–90th percentile: 0.78-1.29) and median signal-to-noise ratio was 0.92. SMRs did not significantly correlate with SEP-1 scores (Spearman’s ρ=0.04, p=0.62) but correlated with pneumonia mortality (ρ=0.27, p< 0.01) and quality star ratings (ρ=-0.29, p< 0.01). Expert and patient panels unanimously endorsed the measure’s face validity in distinguishing hospital performance. Conclusions: CDC’s proposed Adult CO-Sepsis Mortality Measure, based on updated ASE criteria and detailed risk adjustment, demonstrated strong performance across diverse hospital datasets and construct validity through correlation with expected outcome and global quality measures. Its lack of correlation with SEP-1 underscores the need for broader strategies to improve sepsis care and the added value of an EHR-based outcome measure.
Rhee et al. (Sun,) studied this question.