Detection of changes in mortality after heart surgery

Editor—We are concerned about the graphical technique described by Poloniecki et al in their analysis of perioperative mortality rates associated with cardiac surgery.1 Figure 2 shows three traces: observed mortality performance bracketed by control limits and plotted against the number of successive cases performed. The interpretation of the middle of the traces is straightforward since it is simply a variable life adjusted display that has previously been described and will be familiar to many cardiac surgeons in the United Kingdom.2 The use of control limits, on the other hand, is new. However, the usefulness and indeed the validity of these is not clear. As the authors themselves note, their analysis does not amount to a formal test of significance since the control limits have not been corrected for multiple testing; this is a major deficiency. The use of 99% control limits rather than 95% control limits presumably increases their separation and makes them more forgiving. It is not clear which level of significance should be used, a difficulty compounded by the fact that the limits are not based on formal significance testing. If we understand correctly, these control limits have been calculated using a χ2 distribution. However, this fails to take into account case mix and heterogeneity of risk, the very things for which variable life adjusted display plots are used. The following example illustrates the danger in ignoring case mix when estimating ranges of variability. Consider operations on two sequences of 1000 patients with different underlying mortality risks that have been assessed preoperatively (table). Based on the given mortality risks, there is a 99% probability that the number of deaths that actually occur would fall in the range shown in the last column. These ranges are derived from exact calculations based on the binomial expansion. Using a χ2 distribution would give a range (16 to 44) close to the exact values obtained for the patients in sequence 2, for whom no heterogeneity of risk is present, but would substantially overestimate the range for the patients in sequence 1, for whom risks are heterogeneous. When examining surgical mortality, it is important to take case mix into account. However, this should be done not only when estimating the expected mortality but also when estimating the likely variability. Any overestimation of likely ranges of variability might well lead to undue complacency.