Research, audit and journal policies

This month, Steven Shafer, Editor-in-Chief of Anesthesia and Analgesia, sets out that journal’s policy on ethical approval and informed consent in an editorial [1] that is reproduced, in whole, in this issue of Anaesthesia [2] (in order to demonstrate first, how journals can cooperate and second, how Editors-in-Chief can agree to disagree). My invited response – you’re reading it now – is similarly reproduced in Anesthesia and Analgesia, so that readers of both journals can see and join in the dialogue. I am grateful to Dr Shafer and to both publishers for the opportunity and permission to respond. Dr Shafer opens by stating Anesthesia and Analgesia’s requirement for ethical approval and informed consent for research, with which I certainly agree; indeed, it’s a central requirement for this journal too [3]. He then goes on to raise the question: What about audits and case reports? I have addressed this question before in this journal, and Dr Shafer questions my statement that ‘research tells us what works best and audit tells us whether we’re doing it or not’ [4], going on to claim that my distinction is ‘irrelevant’. I both disagree and agree with his claim, as I will attempt to explain below. First, I stand by my definition. If we want to know whether Drug A works better than Drug B for condition X, the process we use is research: we design a study – in most cases, a randomised controlled trial – and from the results, draw conclusions about the relative merits of the two drugs (for the sake of argument, let us say that Drug B is best. And let’s also say that the evidence for this finding is overwhelming and not in doubt). This description meets the criteria for research outlined by Dr Shafer in his reference to dictionaries and the US Department. Audit, on the other hand, is a process by which we ensure that patients with condition X actually do receive the better treatment – Drug B – rather than Drug A. A randomised controlled trial has no place here; we need to look at patients with condition X, either locally or nationally, and see what they are prescribed. We then compare our findings (e.g. that only 50% of such pati'ents do indeed receive Drug B) with our standard of care (this may be a local standard or a national one. Let’s say that the Association of Condition X-ologists has decreed that 90% of sufferers should receive Drug B), and then look at how this sorry state of affairs might be improved. After appropriate intervention, be it education, improved availability of Drug B, better protocols, etc., we repeat the audit in the hope that the figure has increased from the original 50% (‘closing the loop’). The two processes are clearly different, although they both include data collection, analysis and interpretation. Second, I agree with Dr Shafer in that this definition is not useful (which was my point in giving the definition in the first place [4]). The problem is that not all research is so clearly demarcated by a randomised controlled trial, and not all audit fits so well into the ‘audit cycle’ described above. Research is not only about whether one treatment works better than another; it is also about how a treatment works, how a disease process develops, etc. And take, for example, condition Y, which has only just been described; in order to see how sufferers are being treated, we need to collect data in exactly the same way as for condition X, but without the comparison against a standard (for there isn’t one yet). Many would call this a ‘survey’– or even ‘research’; some would still label it an ‘audit’, albeit the first part of one. In the UK, we have additional (and in my view, unhelpful) regulatory descriptors: ‘service evaluation’ and ‘surveillance’, muddying the water further [5]. Hence, my plea that from an ethical point of view, the focus should be on ‘what is being done, not what it is called’ [4, 6]. The studies of condition X and condition Y both involve collecting patients’ data and there are issues of consent and confidentiality that pertain, irrespective of the study’s label. A ‘research’ study exploring the association between patients’ age and white cell count is ethically less potentially harmful than an ‘audit’ or ‘service evaluation’ that looks at the criminal records of nursing staff, even though both only involve examination of existing data. Much of this discussion hinges around the need for ethical approval (in the UK, by a Research Ethics Committee (REC) and in the US, by an Institutional Review Board (IRB)). Dr Shafer proposes that all ‘research’ must have IRB approval and all ‘audit’ submitted for publication is thereby immediately reclassified as ‘research’ and therefore must also have IRB approval. Here, we are both more in agreement than disagreement, and we come down to semantics. From an ethical point of view, it’s generally accepted that although an individual has the right to choose which ‘projects’ he supports, an organisation providing a service can analyse data about its service in order to maintain and improve standards, without necessarily having to ask its customers/clients for permission. Indeed, healthcare organisations have a moral and legal obligation to analyse the service they provide for patients, and patients implicitly accept this when receiving the service. Taking a local audit or survey and publishing it in an international journal, or presenting it at an international meeting, could be argued to exceed that understanding – the promotion of ‘generalisable knowledge’ to which Dr Shafer refers. Dr Shafer states that Anesthesia and Analgesia’s policy will be to insist on IRB approval for all audits before they can be considered for publication, and I can understand that viewpoint – although I do not think that one needs to invoke definitions such as ‘generalisable knowledge’ or ‘systematic investigations’ to reach that position. However, there is another factor that has relevance to readers of Anaesthesia more than to those of Anesthesia and Analgesia: in the UK, investigators are actively prevented from seeking ethical approval for anything other than studies classified as ‘research’ [7]. Since the only bodies (currently) than can give ethical approval for studies in the UK are RECs, and since anything ‘not research’ cannot be submitted to a REC (or to a NHS Research & Development Office either), projects that may have serious ethical issues, but that have the wrong name, are excluded from this process [6, 7]. Furthermore, RECs are not able to grant retrospective approval to a study already done. Thus Dr Shafer’s stipulation that an ‘audit’ can be submitted to Anesthesia and Analgesia only if it has been reviewed by an IRB and considered exempt from further review cannot apply in the UK, since a REC has no mechanism for reviewing it, even to the point of deciding to decline to review it! In that regard, investigators (and editors) should be envious of the situation in the US, where Dr Shafer’s ‘IRB sees all’ approach is more robust and clearer than on this side of the Pond. Both our journals have broadly similar requirements in terms of ethical requirements, although Anaesthesia’s reflect the situation pertaining in the UK (and some other countries too). Therefore, Anaesthesia will continue to consider studies defined as ‘non-research’ that have not been reviewed by the REC or equivalent, where such review is not available locally to investigators. In considering such studies, the Editorial Board will follow the guidance on this situation published by the Committee on Publication Ethics (COPE), i.e. taking into account the study’s scientific validity, presentation, ethical harms and potential benefits (including how these have been minimised/maximised respectively) [8]. This does not mean lower ethical standards, but reflects different regulatory structures. Investigators would be well advised to discuss these studies with their REC chairman (though the latter’s opinion regarding the need for ethical review does not constitute ‘approval’) and their trust’s Caldicott Guardian (in the NHS, the person responsible for protecting patients’ data), or their equivalents, before submitting such manuscripts to Anaesthesia. Our positions on consent are also broadly similar, but Anaesthesia will, rarely, consider case reports where consent (or assent) was not written. In such cases, as for ‘non-research’, each report will be reviewed on its merits, considering its harms, benefits, whether consent/assent could have been obtained, and the implications of the Data Protection Act [9]. I end on something about which we are in clear agreement: the Editorial Board of Anaesthesia, like that of Anesthesia and Analgesia, reserves the right to reject a manuscript irrespective of the local IRB’s or REC’s opinion. Vive la difference! I am Editor-in-Chief of Anaesthesia and a member of COPE Council and of Wiley-Blackwell’s Ethics Advisory Panel. No external funding declared.