Key result
UHC mortality model outperforms STS with C-statistic 0.88, driven entirely by postoperative complications.
Why the study?
The University HealthSystem Consortium mortality risk adjustment models may include postoperative complications in preoperative risk adjustments, raising concerns about their use as quality benchmarks compared to Society of Thoracic Surgeons models.
Does the UHC mortality risk adjustment model provide accurate preoperative risk assessment compared to the STS model in adult cardiac surgery patients?
Observational (n=2,171)
Yes
Does the UHC mortality risk adjustment model provide accurate preoperative risk assessment compared to the STS model in adult cardiac surgery patients?
The UHC mortality risk model inappropriately incorporates postoperative complications into its preoperative risk adjustment, rendering it unsuitable for quality benchmarking in cardiac surgery compared to the STS model.
Answers are grounded in this paper and cite their sources.
UHC models may distort cardiac surgery quality benchmarks by including complications; leaves open whether STS or refined administrative models are preferable.
BACKGROUND: The University HealthSystem Consortium (UHC) mortality risk adjustment models are increasingly being used as benchmarks for quality assessment. But these administrative database models may include postoperative complications in their adjustments for preoperative risk. The purpose of this study was to compare the performance of the UHC with the Society of Thoracic Surgeons (STS) risk-adjusted mortality models for adult cardiac surgery and evaluate the contribution of postoperative complications on model performance. STUDY DESIGN: We identified adult cardiac surgery patients with mortality risk estimates in both the UHC and Society of Thoracic Surgeons databases. We compared the predictive performance and calibration of estimates from both models. We then reestimated both models using only patients without any postoperative complications to determine the relative contribution of adjustments for postoperative events on model performance. RESULTS: In the study population of 2,171 patients, the UHC model explained more variability (27% versus 13%, p < 0.001) and achieved better discrimination (C statistic = 0.88 versus 0.81, p < 0.001). But when applied in the population of patients without complications, the UHC model performance declined severely. The C statistic decreased from 0.88 to 0.49, a level of discrimination equivalent to random chance. The discrimination of the Society of Thoracic Surgeons model was unchanged (C statistic of 0.79 versus 0.81). CONCLUSIONS: Although the UHC model demonstrated better performance in the total study population, this difference in performance reflects adjustments for conditions that are postoperative complications. The current UHC models should not be used for quality benchmarks.
No takes yet. Share an insight, caveat, or question.
Kozower et al. (2009) conducted an observational in Adult cardiac surgery (n=2,171). University HealthSystem Consortium (UHC) mortality risk adjustment model vs. Society of Thoracic Surgeons (STS) risk-adjusted mortality model was evaluated on Model discrimination (C statistic) (p=<0.001). The UHC mortality risk model had better overall discrimination than the STS model (C statistic 0.88 vs 0.81, P<0.001), but its performance dropped to 0.49 when excluding postoperative complications.
Synapse has enriched 10 related papers. Consider them for comparative context: