Scoring complications after congenital heart surgery: gut feelings versus calculations

In times of continuously improving survival rates after congenital heart surgery, it becomes increasingly important to have a more sensitive outcome parameter than death. Insofar, the authors of St Augustin [1] tackle an actual topic in this issue. The heterogeneity of outcome measures and surgical therapies makes it difficult to concentrate on the postoperative care quality in one number. Sata et al. propose an approach similar to that used by the Aristotle score developers. The approach has qualities, limitations and alternatives that are described in the following. Reasons to agree with the proposed scoring system are: Treatment quality is determined by both freedom from death and freedom from complications. The presented paper draws attention to the latter. If many sites register their postoperative complications as proposed, a uniform data collection might result from this attempt. The objective scoring of postoperative complications requires appropriately collected data. If the surgical community will harvest enough data before relevant technical treatment changes make their evaluation obsolete, the paper will be an important contribution to a very useful work. Reasons to be cautious with the proposed scoring system are: The scoring system was generated by experts who used experience and gut feelings to classify the observed complications. The reasons to attribute certain scores to specific complications are not shown in detail. If a spectrum of outcomes ranging from patient discomfort over mechanical ventilation time and permanent neurological damage to economical aspects is condensed into a single number without further information, the score attribution is at risk to be considered arbitrary. Sata et al. apparently sorted complication severity by ranks, which are ordinal (=sorted), non-metric (=intervals between ranks are unequal/not defined) classes. According to statistical conventions, addition and multiplication do not belong to the mathematical operations permitted for this kind of data. But this is proposed in the discussed paper: addition of the scores of all observed postoperative complications, and averaging the observed individual scores. Indeed, averaged ranks can be found often, such as averaged states of New York Heart Association functional classes. This might be done—although incorrect—to give a rough overview about the distribution of functional states of a population. But both, author and reader of such data, must be aware of this being an inexact operation, a result of a mathematical ‘off-conventions’ computation; further calculations with such averages increase the grade of uncertainty and inexactness. Other widely used scores addressed this problem. The APACHE score developers in 1981 explicitly considered the comparative factor weight [2]: (‘In weighting each measurement, the panel sought internal agreement among the weights; that is, a weight of 4 given for a mean arterial pressure of less than 50 mmHg is roughly equivalent to a 4 given for a pH less than 7.15. This rectification of adjustment of weights makes the acute physiology score a cardinal as opposed to a nominal or ordinal scale. This enables us to add the individual weights to reach a final score.’). More recent tools such as EuroSCORE [3] or RACHS-1 [4] use logistic regression methods to calculate (instead of estimating) independent factor weights. Such a derivation of weights substantiates the assignment of score points to certain conditions, making them approximately equidistant. Coincidence of the proposed score with another just estimated scoring system (Aristotle) is not a convincing validation. Instead, the score ideally is applied to a different population, as was done with EuroSCORE and RACHS-1. Lacking a clear endpoint definition (see point 1) makes this difficult. The authors did not show that counting >5 score points was unnecessary. Not each patient with >5 score points is in such desperate conditions that considering further complications becomes evitable. Adding all score points of all procedures performed during an uninterrupted hospital stay is problematic in case of planned staged procedures with occasional interim discharge. The authors suggest to measure the performance of a unit by the formula: Performance = Aristotle complexity score × (1 – morbidity score/5). The concerns regarding equidistance between score points refer as well to the Aristotle score, which results from addition of three sub-scores [5] that have few in common, not even the measuring units. So, the meaning of the factor sum is hard to comprehend. The catenation of such ‘off conventions’ data increases the inherent uncertainty. Defining performance of a unit according to the given formula (Point 6) disregards mortality in the context of performance. This is risky: external institutions, unaware of this inexact wording, are likely to confound this performance with its popular meaning, the unit's general quality. Many sites that use the Aristotle Comprehensive Score enter the status of 50 well-defined complications in a central data base, the EACTS Congenital Database (ECD). It might be more useful to use and—if necessary—to modify the ECD complication list than to install a new list that has to be compiled and maintained. We should reduce rather than inflate the administrative overhead. François Lacour-Gayet et al. [5], interpreting Aristotle and creating the Aristotle score, justified the use of estimations instead of data with a lack of scientific answer. Data were not available then. Meanwhile, information on nearly 130 000 procedures (∼36 man-years’ work) were sent to the ECD; among them, a sufficient number of datasets should contain information on complications. A scientific answer should be available from this large amount of data. The ECD could provide objective information on the following consequences of complications (aspects of morbidity): 'death rate increase before discharge (1) and prolongation of length of stay (2). The weight of temporary complications for complications leading to permanent impairment (3) could also be determined from existing ECD data. Concerning the exact economical impact (4) of complications, a more convincing approach would consist in merging ECS data with cost data of contributing hospitals. The role of complications for long term survival (5) and (6) quality of life is harder to determine objectively. Dedicated studies or an extension of the ECD might lead to this goal.