What should we do when traditional research fails?

“…but I still haven't found what I'm looking for.” U2 Significant resource, in terms of time, effort and cost, is invested in the pursuit of improvements in patient care. A large part of this activity is the identification of interventions that are more effective than existing treatments. This, therefore, begs the question, what should we do when such research activity does not seem to provide the answers we were hoping for? One area that has attracted attention over the past 15–20 years is the use of goal-directed fluid therapy (GDFT) during surgery. However, definitive evidence of benefit has remained elusive, perhaps with the exception of elective high-risk surgery. The paper by Stens et al. in this issue of Anaesthesia adds to the growing number of clinical trials that seem to fail to demonstrate that GDFT has a place in clinical practice 1. Their study investigated the added value of cardiac index and pulse pressure variation (PPV) monitoring to mean arterial pressure (MAP)-guided volume therapy in moderate-risk abdominal surgery. Although the intervention group had a higher cardiac index and lower PPV, there was no difference between the groups with regard to postoperative complications. Why might we be unable to demonstrate that this intervention has value? At what point do we abandon an idea and turn our attention to other areas? The scientific process of discovery tends to start with basic science, moves through in-vitro studies, before being studied in humans and then patients. End-points, and hence our definition of ‘what works’, is dependent on the stage of research, and become more patient-focused as the research progresses. Successful outcomes for in-vitro studies of GDFT tend to investigate changes in tissue oxygenation at a micro-circulatory level 2. However, by the authors’ own admission, a previous study of theirs failed to demonstrate any benefit on tissue perfusion at a micro-circulatory level with the addition of PPV/cardiac index-guided volume therapy to a MAP-based haemodynamic protocol. It is perhaps not that surprising that similar lack of benefit might also be seen at the patient level. An evidence base is not static. In 2011, opinion was weighted towards fluid optimisation as part of an enhanced recovery pathway 3, yet by 2013, doubt had entered 4. Although research into GDFT appears to have been ongoing for a considerable period without reaching a definitive conclusion, there has been greater understanding of the inherent complexity. We have seen liberal, super-optimal, targeted and neutral fluid regimes, variously supplemented by the use of inotropes and vasopressors, and extended into the postoperative period. There is increasing evidence that GDFT has no benefit in laparoscopic surgery, or lower risk patients undergoing moderately complex surgery 5. It is only right that all possible combinations are tried before declaring the intervention a failure. Negative trials are still informative, and help guide future research by better identifying the target population most likely to benefit. Frequentist statistics predominate in medicine, and are characterised by the use of the p value that provides the probability of finding the observed or more extreme result, when the null hypothesis is true. This is different to asking ‘what is the probability that this treatment is effective?’ but is often interpreted as such. This has led to a variety of articles aimed at dispelling myths around interpretation of the p value 6, including the ‘Statistically Speaking’ series in Anaesthesia 7. Additional debate has been growing over the years as to whether frequentist statistics are always the most appropriate approach when deciding on whether a treatment should be adopted, as opposed to Bayesian statistics 8, 9. A key difference is that frequentist statistics give no regard to prior knowledge, unlike Bayesian statistics. Protagonists of Bayesian statistics argue that this paradigm is better placed to help clinicians decide whether there is greater probability that a new treatment will produce a better outcome that the old. Of note, the Optimise trial 10 would have been considered a positive trial that supported the use of GDFT had Bayesian statistics been used 11. Should there be a wider discussion and reconsideration as to whether we are using the correct statistical methods to determine which new interventions we should adopt? It may be challenging to implement, as it would appear that one limitation may be the relatively limited number of Bayesian statisticians 9. The end-points of a clinical trial have an impact on the likelihood that we will find the outcome we desire. Ideally, we need end-points that are deemed important to patients, such as mortality. However, there are diminishing returns from clinical research. Progress within medicine has been such that we have perhaps now reached the limit of what can be achieved by traditional research. The low rate of adverse outcomes within the control arm of a clinical trial means that ever larger trials are required to demonstrate benefit 12. This is certainly true for relatively uncommon end-points such as mortality, particularly following lower risk surgery. The concept of ‘evidence-based medicine’ has been acknowledged as reaching a crisis point. It is increasingly becoming a science of marginal gains which become ever harder to identify. For many conditions, the low hanging fruit of interventions that deliver big improvements have been plucked long ago 13. This leads to the use of surrogate end-points such as a reduction in postoperative complications. However, definitions of complications can vary, which may have contributed to Stens et al.'s results. For some of their recorded ‘soft’ complications, such as hypo- or hypervolaemia, there were no clear definitions or diagnostic tools available, which limited the objectivity of this assessment. Initiatives such establishing core outcome measures in peri-operative and anaesthetic care (COMPAC) will help with standardising end-points of clinical trials that will benefit not only individual trials but also make subsequent meta-analyses more robust 14. The conduct of such trials in specific high-risk groups with a high control arm event rate, or looking for specific end-points, may be better able to answer questions about the role of GDFT. Optimise-2 is currently underway, with the intention of recruiting 2500 patients to specifically look at postoperative infections as an end-point 15. It is evident from the first two years of National Emergency Laparotomy Audit (NELA) data that there is equipoise over the use of GDFT in emergency laparotomy, with its use remaining unchanged in around 54% of patients 16. The forthcoming multi-centre FLOELA trial 17, aiming to recruit 7500 patients to investigate GDFT in emergency laparotomy, should therefore help add to this evidence base. If we cannot detect benefit in isolated use, what about use within care bundles that have become more prevalent? Our ability to show that care bundles improve patient outcomes arises in part from the aggregation of marginal gains seen in individual interventions that are singly difficult to prove effective. One of the characteristics of effective care bundles is that they collate together a seemingly disparate group of interventions. Although Stens et al.'s study ‘bundled’ different things together, they were perhaps too similar to benefit from this aggregation of marginal gains. Pulse pressure variation, cardiac index and MAP-based cardiac output monitoring are all essentially targeting the same area. Contrast this with elements used in bundles for ventilator-acquired pneumonia 18, enhanced recovery programmes 19 or emergency laparotomy care 20. However, there remains the issue of teasing out the effective versus the ineffective element of the bundle. The favourable effects of goal-directed therapy on outcome only appear to be present in settings where patients were not managed on an elective enhanced recovery pathway 21, most likely because the patient is fluid replete at the start of anaesthesia 22. This creates an interesting quandary: GDFT does not work within ERPs, yet GDFT is included in many such bundles. It may be that the science behind GDFT is sound, but the ability to deliver this in a reliable fashion in a clinical environment is the weak link. This is the key difference between the effectiveness of an intervention, as opposed to its efficacy. Following a power calculation, Stens et al. successfully recruited 244 patients, but only 175 were eligible for analysis. This may have meant that the study was underpowered, but there is an additional implication of this failure rate. The ~30% drop-out rate (6% due to technical issues with the monitor) is considerable, but is even more striking when one considers that adherence to protocols around delivery of care is usually more reliable in the context of a clinical trial, compared with the relatively uncontrolled clinical environment. The efficacy of an intervention is irrelevant if it is not possible to deliver it in a reliable fashion (effectiveness). This is not necessarily a call for more studies of effectiveness as opposed to efficacy, as Stens et al.'s intention to treat analysis essentially produces an effectiveness study. Rather, we need more studies that tell us how to better implement interventions. Given the above issues, we have perhaps reached the limit of improving patient care by investigating what we should do, and now need to turn our attention to how we should do things. Regardless of the advance in medical science that is better able to define potential improvement, the reality is that we still struggle to reliably deliver healthcare for ‘big’ interventions. For instance, the NELA data shows that around 15% of patients with a septic indication for surgery only received their first dose of antibiotics in the operating theatre, although presumably this would have been evident at least at the time of decision making (data to be published in the third NELA report). Addressing these failures requires greater understanding of both the ‘workspace’, and implementation science. Healthcare is delivered in a complex environment. An area that is currently under-explored in healthcare is the application of human factors and ergonomics to improve the design of the system such that human performance is optimised 23-25. It is important to appreciate that human factors are not just about crew resource management and non-technical skills training, as is so often implied by various training courses. Human factors and ergonomics describe the interactions between all aspects of the wider workspace: people; the physical environment; tools and technology; and the processes of care that we engage with in the delivery of healthcare 26. Application of human factors and ergonomics allows not only better understanding of error prone activity, such ‘wrong blood in tube’ incidents 27 but can also be used to improve the design of processes and the workspace in which they function. For instance, human factors and ergonomics methods have been used to better understand the workload within anaesthetic rooms 28, 29 and other areas 30, and provide design recommendations to make tasks easier. There are a variety of studies that have successfully used either quality improvement methodology, or implementation science such as normalisation process theory to embed interventions into clinical practice 31-33. Both methodologies share similarities with human factors and ergonomics 34. Tengberget al. demonstrated a reduction in 30-day mortality from 21.8% to 15.5% using a care bundle, which included GDFT and a standardised anaesthetic protocol, in 600 patients undergoing emergency laparotomy 35. Cannesson et al. found a reduction in length of stay and complications for patients undergoing high-risk abdominal and pelvic surgery following the successful implementation of a care bundle that included a standardised goal-directed fluid protocol 36. Moore also used a similar approach to embed a care bundle into an existing enhanced recovery pathway that was successful in reducing postoperative pulmonary complications from 19.3% to 10.5%, which was sustained at one year 37. These types of study are characterised not so much by the intervention and comparison with historical controls, but the accompanying effort to embed the interventions into clinical practice through the use of team-directed training, measurement of adherence to protocols and feedback to staff, as well as patient-mediated interventions. The EPOCH study is due to report soon, and should aid our understanding of quality improvement methodology in promoting the implementation of a peri-operative care pathway for patients undergoing emergency laparotomy 38. Another characteristic of such implementation studies is that they aim to standardise delivery of care, often through the use of protocols. In the absence of evidence as to what constitutes a ‘better anaesthetic’, there may well be improvements brought about by ensuring that we do something, but we do it well. The recent focus on standardisation of anaesthesia for hip fracture has been controversial, due to the perceived threat to professional autonomy 39. Yet this autonomy is maintained by using our expertise to decide when a patient does not warrant standardised care. In the meantime, the remainder of patients benefit from reliably receiving all other elements of care. If we do standardise care, and this produces better outcomes than non-standardised care, then surely we have achieved something that has so far eluded us by attempting to define the marginal gains of whether technique A is better than technique B? The additional benefit is that standardisation of care also provides a more stable baseline from which to measure additional change using implementation studies. Approaches such as these should help reduce the estimated 17-year time lag in translational research 40 by addressing this second translational gap that covers the delivery of healthcare to patients 41. Huge advances in care have been realised by the development of clinical research. However, as modern medicine becomes ever more successful in improving patient outcomes, this ‘traditional’ approach to research appears to be less able to offer the advances we desire. There is arguably far more to be gained by establishing how to reliably deliver existing care than there is chasing ever decreasing marginal gains. This will require a change in mind-set as we embrace different research paradigms, but the potential improvements for patients are still there to be realised. DM is the national clinical lead for the National Emergency Laparotomy Audit. No competing interests declared.