Providence, patient or provider? Looking for truth in retrospective database studies

As clinical practitioners, it is easy to assign significance to a single event. All of us will have experienced a patient desaturating following tracheal extubation. There are multiple possible causes, ranging from easily avoidable problems such as residual neuromuscular blockade or anaesthesia, to the more inevitable and multifactorial such as age, morbidity and intercurrent illness. Deserved or not, we may blame ourselves for these events, and feel perturbed about the diversion from a smooth and professional looking finish to the case. Subsequent events depend on the nature and cause of the event, but assuming the patient does not require tracheal re-intubation and transfer to higher level care, then many of us will have some degree of anxiety for the patient in the post-anaesthetia recovery ward. Could this single event lead to a worse long-term outcome for the patient? And, moreover, could this event be avoidable? It is this thorny issue that Rostin et al. try to resolve in this issue of the journal 1. The authors used a large single-centre registry database to investigate the effect of early desaturation (below 90%) following tracheal extubation in patients undergoing inpatient surgery. They aimed to look for an association between this binary outcome and a variety of adverse postoperative events, with their primary end-point being effect on discharge destination. The authors should be congratulated for a number of reasons. Their study, although being single-centre, included an excess of 70,000 patients. They chose an outcome which is easily measured and clearly of significance to patients. The premise of this study also provides an interesting counterbalance to recent studies in the critical care literature that have suggested a potential benefit in a restrictive approach to oxygen therapy, whereby lowering the resultant oxygen target and avoiding excessive oxygen tensions (so called ‘hyperoxia) is beneficial in a variety of acute illnesses 2, 3. Methodologically, they have also undertaken multiple sensitivity analyses to try to test the robustness of their findings and made every effort possible to address the problems inherent to a retrospective registry study. However, before accepting their findings, we should consider their validity. In their conclusion, the authors assert that an episode of desaturation is a target for potential therapy and that preventing this occurrence could reduce the incidence of adverse discharge. Although this is attractive, is it biologically plausible? Could a single short-lived desaturation make a difference to such a profound and long-term outcome? And if so, by what mechanism does this occur? We would suggest initial caution in the wider interpretation of these findings beyond hypothesis generation. Although the association of transient hypoxia (median duration 1.6 min) with discharge destination is an interesting finding, the effect of such a short-lived event on ultimate discharge destination does not seem biologically plausible when considered in isolation. Although the authors have made a considerable effort to correct for confounders, we suspect that their results may reflect an inability to correct for all those possible. This should not be considered as a criticism of the authors, but rather an acknowledgement of the limitation of a retrospective analysis. In all analyses, the finding of an independent association between desaturation and adverse discharge outcome was consistent. However, there are still multiple potential interpretations. First, that there is a true independent association between desaturation and adverse outcome. Second, that there are unknown confounders (e.g. frailty) within the groups that are contributing to the outcome. Measures such as this could be very difficult to assess retrospectively. We contend that such patients may be more likely to desaturate, develop complications and thus more likely to be admitted to long-term care postoperatively 4, 5. Although intuitively the latter interpretation appears more likely (at least to this commentator), this does not diminish the importance of the authors’ findings. Desaturation after tracheal extubation is clearly associated with worse postoperative outcomes and could be a useful marker for postoperative intervention trials, even if the desaturation itself is not the target of the study. These potential interventions could include admission to a higher level of care such as the ‘advanced recovery room care unit’ being trialled for medium-risk surgical patients (consisting of a longer duration of recovery care, higher nursing ratios compared with ward care and more frequent medical review), or more specific respiratory interventions (Australian New Zealand Clinical Trials Registry number 12617001173381). It may be possible that hypoxia after tracheal extubation is an early manifestation of a postoperative respiratory complication. These complications have been consistently associated with increased hospital morbidity and adverse outcome in the literature 6. Given their greater longevity, this may be a more plausible determinant of discharge destination. We, therefore, do agree with the statement by the authors that it is more likely that this desaturation represents an association with vulnerability to later desaturation (or even more likely, the sum of other unmeasured comorbidity or postoperative complications). Could the desaturations have been avoidable? This is a very tricky question to answer in the context of a retrospective database study. There are some notable associations worth discussing. The mean intra-operative morphine-equivalent dose (34 mg), although a reasonably large dose, may have been appropriate to the clinical situation. It would, therefore, be interesting to know what group of patients were receiving these intra-operative doses and whether other analgesic options were considered and given. Additionally, it would be interesting to know when patients were receiving the higher neostigmine doses. Was this in response to hypoventilation and desaturation? Or a second dose of reversal due to residual neuromuscular block 7? The authors’ results indicated that ‘lung-protective ventilation’ was found to be protective against early hypoxia in this study but the definition (a plateau threshold of 16 cmH2O) is not consistent with previous definitions 8. Lung-protective ventilation is traditionally defined as a low-volume, low-pressure strategy of mechanical ventilation which consists of 6–8 ml.kg−1 lean body mass tidal volume and a target pressure of < 30 cmH2O. Although plateau pressure > 16 cmH2O has been shown to be associated with pulmonary complications in another study using the same database, it is not known whether lower pressures are truly ‘protective’, or whether this simply represents an association 9. As is often the case, it is hard to judge whether these areas truly represent avoidable insults without case-specific information. One finding which we should be particularly circumspect about is that of the difference between anaesthetist groups. The authors’ results indicated that one of the associations with adverse outcome was the presence of a supervised anaesthetic resident vs. a nurse anaesthetist. With regard to this, we firstly note a p value of 0.045; this is a fragile difference. Were there other groups tested, what was the measure of error within providers and were there significant differences between individual anaesthetists (beyond the descriptive 3.76–4.67% quoted)? Given the multiple outcomes tested in this manuscript it is plausible that this is a chance finding. Second, and perhaps more importantly, we should cast our minds back to 2015. An article described a study that had found a significant difference between individual cardiac anaesthetists and patient outcomes. This was greeted with considerable excitement: “At last, something to show that I'm better than that anaesthetist next door, I knew it!”; or, for the more kind minded, “We can do something that makes a difference, I knew it!”. Unfortunately (depending on your perspective), these findings were criticised on multiple grounds. It was later found that there was no significant difference between anaesthetists, and the article was retracted 10, 11. We suggest that this finding may possibly reflect a difference in case-mix or the severity of patient comorbidity. It was not clear from the manuscript how the cases are assigned to the different groups, and without knowing this (and without random allocation) it is hard to fully discount the effect of case assignation bias. It should also be noted that ranked plots are not the fairest or most effective means of comparing providers and that no measure of certainty is given for these values 12. Finally, the assertion that variability in practice and outcome rate suggests preventability is not entirely cogent and needs to be demonstrated experimentally. The authors suggest focusing on the implementation of tracheal extubation protocols after surgery in order to prevent early complications, and refer to their own quality improvement study on the topic of optimal neuromuscular blockade management 13. This is an area which could be considered to have been relatively neglected in the literature to date despite being considered a period of high risk for adverse outcome 14, 15. Whether such an implementation will prove successful in improving patient outcomes would be of significant interest. The linkage of large clinical and administrative databases provides a very powerful research tool. The data available can provide a useful insight into disease states and their associations. Nonetheless, the studies are prone to the confounding factors we see in other observational research. Even with the best intentions, it is not possible to entirely discount the possibility of unknown confounders leading to a non-causal association. Very promising findings from observational trials have been disappointing when followed up by randomised controlled trials 16. We should, therefore, remember to treat these findings as hypothesis generating rather than definitive. Where possible we should follow these studies up with a prospective trial to try and determine where the truth lies. This becomes difficult when randomisation is unethical. Clearly, we cannot randomly allocate patients to a period of desaturation after tracheal extubation. However, we could consider nesting interventions within these datasets. The authors discuss improvements in training to attempt to improve the desaturation rate. Could this improve the adverse discharge rate? A standard randomised controlled trial would be probably be prohibitively impractical. However, a novel trial design, such as a stepped-wedge nested within registries could be possible and may bring us closer to an answer. No conflict of interest declared.