Key result
This editorial discusses the need to update and recalibrate the EuroSCORE risk model for cardiac surgery, suggesting that institutions adjust the logistic EuroSCORE based on their own performance.
Answers are grounded in this paper and cite their sources.
Institutions outperforming EuroSCORE predictions should adjust the logistic EuroSCORE by their own performance ratio for accurate risk assessment and quality control.
In this issue [1], Nissinen and co-workers have attempted to improve the accuracy of EuroSCORE (logistic and additive) by developing a new model which takes some of the EuroSCORE variables and modifies others. This is based on fewer than 4000 patients in one institution. The authors have been accurate in identifying the areas where EuroSCORE can be improved (lack of sufficient attention to the ‘weight’ of the intervention, use of simple creatinine rather than estimated glomerular filtration or creatinine clearance for assessing renal function, some relatively subjective variables, etc.). They have produced a new model which has better calibration and very slightly better discrimination when compared to EuroSCORE as applied to their patient population. Although the paper purports to recommend improvements to the accuracy of EuroSCORE, it is in fact a new proposed risk model, and it therefore should be approached with rigour if it is to be taken seriously by the cardiac surgical community. The first issue is that, as single-institutional study, the results could be telling us more about the institution than about the model unless tested elsewhere; the second is that I could find no evidence of a developmental dataset and validation dataset, which is the standard approach for testing models before proposing them. If the model was derived from the same patients on whom it was tested, it is not surprising that it performed well. Clarification of this point is essential, although the modifications suggested by the authors are intuitively sound. More importantly, however, the authors have confirmed a finding that is relevant to many cardiac surgical institutions: EuroSCORE, developed more than 10 years ago, may now be out of date. There is no doubt that several institutions now outperform EuroSCORE predictions, but it is difficult to be sure of the widespread prevalence of such overperformance. In countries with national data collection and publication, such as the United Kingdom (http://heartsurgery.cqc.org.uk/), recent figures suggest that UK cardiac surgical mortality is currently running at approximately 0.6 of EuroSCORE prediction. On the other hand, I have direct and reliable data from institutions where cardiac mortality is as predicted by EuroSCORE, and until last month have reviewed papers for this journal and others in which authors stated that for their institutions, EuroSCORE underpredicts mortality. Understandably, more publications tend to come from those units that perform better rather than worse when compared with EuroSCORE predictions. Nevertheless, there can be no doubt that the questions about the calibration of the model are valid and can now only be answered by a new and major data collection exercise. I am delighted to announce that the funding for this project has finally been secured and that we anticipate a start date later this year. Several units within and outside Europe have already registered to participate in anticipation of this project, and other interested units are invited to register their interest by contacting euroscore@papworth.nhs.uk. In the meantime, what should units that outperform EuroSCORE do for risk assessment and quality control? The answer can be remarkably simple. Most reports, including this article, indicate that the model remains powerfully discriminatory (area under the ROC curve in this article is >0.8), so that the main problem faced by many units is that of calibration. Any reasonably calibrated risk model offers a set standard. Some units will perform at that standard, some will do better and some worse. The best estimate for evaluating the risk of mortality for a patient undergoing a particular procedure at a particular institution is to use logistic EuroSCORE and then correct it for the performance of the unit in question, so that the patient should be quoted a predicted mortality calculated as follows: In other words, in Hospital X, where actual mortality is, say, 0.7 of predicted, the patient’s logistic EuroSCORE is multiplied by 0.7, reflecting the hospital’s performance. A corollary of the above is where risk modelling is used in defining the cut-off points for the evaluation of high-risk procedures, such as percutaneous aortic valve replacement (pAVR). If it is decided in Hospital X that pAVR should only be considered for patients with a predicted mortality of 20% or more from conventional surgery, then, taking the hospital’s own performance into account, pAVR can be offered at Hospital X for patients with a logistic EuroSCORE of 28.6 or higher (20/0.7 = 28.56). Finally, an increasing number of cardiac surgical units employ risk modelling to define the standard acceptable target for clinical performance using, amongst others, risk-adjusted hospital mortality as a surrogate indicator for quality. These targets may need to be adjusted downwards in a similar way in order to maintain the impetus towards continued quality improvement.
No takes yet. Share an insight, caveat, or question.
Samer A.M. Nashef (2009) conducted an editorial in Cardiac surgery. EuroSCORE risk model was evaluated. This editorial discusses the need to update and recalibrate the EuroSCORE risk model for cardiac surgery, suggesting that institutions adjust the logistic EuroSCORE based on their own performance.
Synapse has enriched 10 related papers. Consider them for comparative context: