Creating guidelines and treating patients when there are no trials or systematic reviews

Readers will be familiar with the tenets of evidence-based medicine.1 The process can be summarised in four steps. First, the clinical problem must be formulated into an answerable question. Second, the best evidence to answer the question must be located. Third, the evidence must be appraised for validity, relevance and applicability. Fourth, the results must be implemented in practice. Within the usual ‘hierarchy of evidence’, a systematic review of randomised controlled trials (RCTs) is said to provide the soundest foundation to support clinical decision-making, with a single RCT the next best source of evidence. Observational studies and case reports make up the lower levels of the hierarchy. This is now well known but needs to be tempered by two caveats. First, although RCTs are the preferred study design for questions about the effectiveness of interventions, they will not always be possible for ethical or logistical reasons. Other designs are better suited for other types of question. It is also true that the nature of patients recruited into trials, and the carefully controlled conditions under which trials are conducted, may make the results less applicable to patients in general. Second, the hierarchy holds true when the evidence is of high enough quality, but not necessarily in every case, as a well conducted, relatively unbiased large observational study may provide more compelling evidence than a group of small poorly conducted RCTs. Although, for a number of reasons, no field of medicine is fully evidence-based, this might be more true for anaesthesia and critical care than many others.2 Paradoxically, although doctors have practised for centuries without what we regard today as research evidence, it is only recently that such evidence has become available, so that now, practising without it can be considered a problem. Creating guidelines Clinical guidelines are a useful tool for getting research evidence into practice, as their production includes all four steps of the evidence-based process for a given clinical question or set of questions. Clinical guidelines, thus, have an important role to play within the promotion of evidence-based practice. They are defined by the WHO as ‘systematically developed evidence-based statements which assist providers, recipients and other stakeholders to make informed decisions about appropriate health interventions.3 The definition implies a careful search for relevant literature, the use of approved measures for appraising the validity and applicability of the findings and explicit procedures for forming recommendations. A useful template for the guideline production process is the Appraisal of Guidelines Research and Evaluation-II (AGREE-II) instrument, which comprises a 23-point checklist covering scope and purpose, stakeholder involvement, rigour of development, clarity, presentation, applicability and editorial independence.4 Guidelines may be difficult to apply to some patients – for instance, those with comorbidities – as there may be less or no evidence for this type of patient. They may also be produced for ‘non-clinical’ reasons; guideline production confers prestige and authority on the organisation that issues them.5 Thus, it is common for guidelines to be produced even when evidence is lacking. Guideline production needs evidence, but may also require expert opinion to set the research evidence into its clinical context and produce an authoritative recommendation. Both evidence and opinion have their place, but I believe that it should be possible to distinguish between them in the finished guideline. Experts commonly disagree, especially when the evidence is lacking, and this should also be made explicit. An important point too is that all those involved in the production process should declare their interests. It is almost inevitable that those who are well known in their field will have made a substantial personal investment in their academic career and many will have some industrial connection too. These should be declared. It could be argued that it is precisely when evidence is lacking that authoritative guidance is needed. In these circumstances, the potential for conflicts of interest to bias expert judgement may be greater too, so extra transparency is also necessary. (A recently published guideline from the European Society of Anaesthesiology has tried to attempt exactly that – to focus on topics which are relevant to practice, but which are not well covered by the literature.)6 Treating patients When we begin to learn clinical anaesthesia, we first learn a sequence of practical activities, developing proficiency in each as time goes on. It is only later that we begin to acquire the ‘evidence’, the underlying knowledge which supports these actions. But what form does this ‘evidence’ take? It may be based on argument from first principles in physiology or pharmacology, or the unequivocal support of our own senses, or a large analysis of past problems drawing lessons for practice. More rarely, there is a published research study of some description to guide us; more rarely still, a RCT or systematic review. So what do we draw on when we treat patients day by day in the operating theatre or intensive care unit? Knowledge theorists postulate two types of knowledge:7 ‘explicit’ knowledge, which can be readily written down, codified and communicated in textbooks and journals and set out in examination syllabuses; and ‘tacit’ knowledge, defined as ‘knowledge that has not been (and perhaps cannot be) formulated explicitly and, therefore, cannot be stored or transferred entirely by impersonal means’.8 It is typically acquired by demonstration followed by practice. To take regional anaesthesia as an example, there are to my knowledge no RCTs of dealing with different methods of performing ‘difficult’ spinal anaesthetics, for example. This is such a commonly performed procedure that one might find this lack of scrutiny surprising. However, this does not mean that there is no knowledge to guide our practice, but rather that the knowledge must come from other sources than RCT. Our observational study of regional anaesthesia uncovered a number of elements which had no research backing but undoubtedly contribute to the effectiveness and safety of expert practice.9 We suggested that experts acquire a whole repertoire of such techniques and are able to draw on these as required, selecting those which apply to a particular patient. In a similar way, we ‘individualise’ the results of a RCT to a particular patient. Once one is aware of tacit knowledge, one sees it everywhere. It lurks behind the apparently objective findings of research and plays a key part in what we do, for instance in the spoken ‘routines’ anaesthetists use to talk to patients and colleagues.10 A further example illustrates this. In 2002, a hospital in the United Kingdom dealt with the largest outbreak of Legionnaires’ disease in the United Kingdom to date. An ‘early warning scoring’ system for identifying deteriorating patients had, by chance, just been introduced into the hospital. Retrospectively we were able to retrieve both the vital sign observations and the scoring charts and calculate the error rates in scoring.11 Not only was the error rate higher in patients with confirmed Legionnaire's disease than in those with similar symptoms and negative serology (17 vs. 12%, P = 0.02), but seropositive patients were consistently overscored, making them appear sicker than they actually were. This led us to surmise that information over and above that derived from patients’ vital signs (the ‘clinical acumen’ of experienced staff) was subconsciously influencing the allocation of scores. Given that early warning scoring systems appear to be robust predictors of outcome,12 it is important to fully understand the scoring behaviour and perceptions of staff and understand how these play into risk prediction.13 This would be a welcome subject for further enquiry. Conclusion First, it is not only possible but also desirable to treat patients without recourse to RCTs. Even when trials are available, there are reasons why it is not appropriate to apply their results to a particular patient. In fact, in knowledge terms, we proceed in a very similar way whether there is an RCT available or not. What differs is the type of knowledge we draw on in making our decision about the patient. Second, guidelines can and should be produced when there is no ‘high-level’ evidence, subject to the caveats above. Transparency of process and better methods for harnessing expert opinions within groups will help with this and as such guidelines can become high-quality, practical ‘descriptions’ of what works in practice as much as ‘prescriptions’ for what ought to happen that are based on research evidence. Finally, although Archie Cochrane, the pioneer of evidence-based medicine who gave his name to the Cochrane Collaboration, is best known for his scientific approach to healthcare, even he knew that, when there is no scientifically based treatment available, kindness, compassion, communication14 and humanity are irreplaceable in medical practice. It is impossible to excel in anaesthesiology without bringing these elements to one's professional life and work.15 Acknowledgement Assistance with the study: none declared. Financial support and sponsorship: AS is in receipt of a personal patient safety research development grant from the UK National Institute for Health Research. He is also an Honorary Research Fellow in the University of Manchester, UK. Conflicts of interest: AS was Chair of the ESA Guidelines Committee from 2009 to 2012 and is an editor in the Cochrane Anaesthesia Review Group. Comment from the editor: this editorial is based on a refresher course held at the Euroanesthesia Meeting, Barcelona 2013. The article was checked and accepted by the editors, but was not sent for peer review.