Key result
Retrospective validation of EuroSCORE II over a 5-year period may be misleading due to dynamic changes in patient characteristics, case-mix, and baseline risk.
Answers are grounded in this paper and cite their sources.
Retrospective validation of risk models like EuroSCORE II over wide time periods may be misleading due to temporal shifts in patient characteristics and case-mix.
We congratulate Chalmers et al. [1] on their recent article that demonstrates the potential validity of EuroSCORE II. As a result of numerous contemporary external validations, it is now widely accepted that the original EuroSCORE is obsolete for contemporary cardiac surgery [2]. Before EuroSCORE II can be adopted for either future clinical or governance applications, it is important that it is independently validated. Chalmers et al. used data from a single unit between January 2006 and March 2010 for their validation study. We would, however, raise a note of caution around the study period. EuroSCORE II was developed on data from patients who underwent surgery between May 2010 and July 2010 and is presumably intended for prospective use [3]. The most recent data included in Chalmers’ validation study is over 2 years old, and performing model validation over a time period as wide as 5 years is potentially misleading. In the same way that the original EuroSCORE models have systematically drifted from the acceptable levels of model calibration with increasing time, reversing the arrow of time might have a similar shifted effect due to dynamic changes in the patient characteristics, case-mix and baseline risk relative to those in the EuroSCORE II patient cohort [4]. Hence, if the model performed badly in 2006 but adequately in 2010, then mixed inferences might be drawn in an overall study period validation. Also, to add clarification to Chalmers’ paper, one of the models analysed in this study was a modified EuroSCORE model developed by the Society for Cardiothoracic Surgery (SCTS) in Great Britain and Ireland. This model was a complex recalibration of the original logistic EuroSCORE updated using registry data collected between 2004 and 2007. The modified EuroSCORE, like the original EuroSCORE models, now demonstrates poor calibration for contemporary cardiac surgery. The SCTS has recognized this and consequently no longer uses this model. Interestingly, despite model calibration deteriorating over time, discriminatory ability has remained consistently good. We also note with interest that Chalmers et al. demonstrate different results in the isolated aortic valve replacement subgroup from those in a separate large multicentre validation study that includes all NHS hospitals in the UK [5]. Namely, the c-statistic was 0.69 (95% confidence interval [CI] 0.57–0.79), which is below the 0.7 threshold for acceptable performance. However, in the national study, it was 0.77 (95% CI 0.67–0.77), which is considered acceptable. Differences in the results for model validation studies within subgroups could be attributable to variation between units and analytical subtleties. Nevertheless, these important patient subgroups warrant further investigation, possibly with a view to introducing procedure-specific models.
No takes yet. Share an insight, caveat, or question.
Hickey et al. (2012) conducted a letter in Cardiac surgery. EuroSCORE II was evaluated. Retrospective validation of EuroSCORE II over a 5-year period may be misleading due to dynamic changes in patient characteristics, case-mix, and baseline risk.
Synapse has enriched 10 related papers. Consider them for comparative context: