The RCT: a very beautiful technique

Taking my lead from those who have presented before me, I understand that the ethos of the Janet Kinson Lecture is that ‘it enables reflection on areas that are important to the speaker, and in so doing demonstrates their education and learning’.1 I have worked in diabetes education and research over the last 15 or so years, and the subject chosen for the lecture at the 2009 Diabetes UK Annual Professional Conference was derived from these experiences. Clinicians are expected to use evidence as a basis for practice whenever possible and randomised controlled trials (RCTs) are held up as the gold standard when designing research to test interventions. While the strengths of the design are not disputed when testing new medications, diabetes management employs a wide array of activities with a behavioural component, and these interventions must also be evidence based. However, there are many challenges when designing studies to investigate interventions with a behavioural component. My presentation focused on some of the issues to be faced when designing RCTs to determine the efficacy of behavioural interventions, and drew on examples from several RCTs in diabetes in which I was involved. The ways in which these issues may compromise external validity will also be considered. Traditionally, health care was based on conventional practice, local wisdom and personal preferences, but the emergence of clinical governance endorsed the need for evidence-based practice. Three crucial factors in health care delivery today are patient safety, meeting targets and cost effectiveness. Therefore, the need to know whether clinical care is effective and economical was never more important. When the focus of care was on pharmacological treatment there was often a wealth of evidence to draw from, but when thinking about many aspects of nursing practice – for example, patient assessment, education, promotion of self-management or empowerment – the evidence base was much more elusive. My own work has been to promote research to underpin nursing practice and this led to my interest in RCTs and the results that they may generate. The evidence base for nursing needs to be derived from properly conducted scientific research. While there are many research methods available, experimental methods – and, in particular, RCTs – are generally recognised as having the necessary rigour to inform clinical decision making. It has been said that ‘The “gold standard” research method for addressing what works in evidence-informed policy making and practice is the randomised controlled trial’.2 The basic framework for conducting an experiment is simple (Table 1).3 As described by Pawson and Tilley,4 you take two evenly matched groups, measure the specific attributes that are of interest, treat one group with the intervention but not the other, then measure the groups again and compare the changes between the two groups. If the two groups are identical at the start and the only difference is the intervention given to the one group, it can be assumed that any differences must be due to the intervention. ‘Lo and behold you have a clear measure of the impact of the program.’4 The first randomised experiments were conducted in agriculture in which the experiments were based on plots of land to which different crops or fertilisers were assigned.5 Pocock5 points out that this enabled the experimenter to have all units available at one time and to tightly control the situation. Randomised experiments to evaluate new treatments gained increasing credibility and became recognised as ‘the only reliable basis for evaluating the efficacy and safety of new treatments’.5 Despite the agricultural origins of this design, one of the great champions of this method was Dr Archie Cochrane who established a register of RCTs in health care research to enable health care professionals to access robust research in order to inform their practice. This was eventually developed into The Cochrane Library that provides ‘high quality, independent evidence to inform health care decision making’.6 The strengths of RCTs are not in question when, for example, testing out a new medication in which one group takes the new product whilst the other group takes the placebo. However, RCT methods can prove difficult to design in the case of complicated behavioural interventions, many of which are at the heart of clinical practice in diabetes. This is not a new challenge – indeed, the title of this Janet Kinson Lecture was based on a quotation from Dr Cochrane who stated: ‘The RCT is a very beautiful technique, of wide applicability, but as with everything else there are snags.’7 Six areas prone to ‘snags’ are outlined below. In the words of Lindsay,8 a complex intervention may be characterised by ‘…︁ actions that are difficult to define and by varied, and difficult to control, contextual factors’.8 Such characteristics would often apply to behavioural interventions designed to improve diabetes management. For example: ‘Do diabetes specialist nurses (DSNs) make a difference to diabetes outcomes?’, ‘Are nurse-led clinics as effective as consultant physician-led clinics?’, and ‘What are the effects of patient education on diabetes management?’ Complex interventions are influenced by multiple related and interdependent variables such as practitioner behaviour, patient behaviour, timing and frequency of behaviour, organisational issues, the setting and location, and local culture. When conducting an experiment, clearly defined and delineated behaviours usually need to be the focus in order to provide the necessary levels of accuracy of interpretation and measurement. Yet, to do so may require the complexity of diabetes management and its susceptibility to a range of external influences to be tightly controlled and for other factors to be excluded from the research design. Standardising procedures, a fundamental criterion of the RCT, is virtually impossible when highly complex treatment interventions are involved. As an example, Glasgow9 presents a pyramid of social-ecological perspectives that may both influence and impact upon outcomes relevant to diabetes self-management education. The pyramid includes what he terms formal spheres of influence such as health care systems, work sites and policy actions, as well as informal ones including social, physical and environmental factors. All these factors may influence the behaviour of an individual involved in a clinical trial and, conversely, a trial would need to take into account an array of outcomes, not only the physiological ones that tend to be most readily measured. The inclusion of such an array of variables was not envisaged when the initial experimental design was developed and in which the only item to change was the intervention itself. In an experiment all contextual variables are controlled, but in the health service this is difficult – indeed, it would be an over-simplification to assume that all contextual variables can be controlled in the real world. In my own locality (Northern Ireland) we have just completed a major re-organisation of the health service in which 18 health and social care trusts have been amalgamated into five trusts. Imagine the effect that this would have upon an experiment, if midway through the study the entire organisational structure were to change and with it trust boundaries, local practices and possibly locality of service provision. Another form of contextual influence is that of policy change. Within the world of diabetes care a good example is that of the new General Medical Services contract and the Quality and Outcomes Framework in which GPs were remunerated according to the extent to which quality targets were achieved. This policy led to an improvement in the average HbA1c across the country. If you were mid-trial with a study designed to improve HbA1c when this policy was introduced, it would be impossible to say whether differences detected – or, indeed, not detected – between the two groups were due to changes in policy rather than due to the effects of the intervention per se. Another example of the changing context of care was encountered in the middle of an RCT (the ESMON study) regarding the effects of self-monitoring of blood glucose in those newly diagnosed with type 2 diabetes.10 A major marketing campaign was launched by one of the medical devices companies to promote the use of blood glucose meters on breakfast television. This meant that patients who had been randomly allocated to the control group then were influenced by the advertising campaign and wondered if they should have been given a meter, and some asked if they could switch to the other group. This issue was overcome, but it serves to illustrate that even with a carefully planned study it is not possible to predict the impact that outside influences may have upon the subject under scrutiny. It is generally recognised that: ‘A clear definition, the ability to control outside factors and standardisation of the intervention are cornerstones of the RCT’.11 The ability to define clearly what exactly is being investigated has often proved to be a challenge in behavioural interventions. If we consider some of the issues that are of relevance to diabetes management – such as self-care decision making, self-management, patient empowerment, depression and anxiety, socio-economic position, and quality of life – it can be seen that either the nature of the intervention or the outcomes could be open to interpretation; therefore, they need to be carefully defined in the context of the study. The difficulty of clarifying such variables was illustrated in a literature review by Paterson et al.12 to investigate the assumptions underlying the traditional conceptualisation of self-care decision making. They found that the concept was not static, but that it waxed and waned and was influenced by many factors. Such abstract concepts need to be clearly defined if the results are to be properly understood and if the intervention is to be replicated. While this may sound like an obvious point, an analysis of 47 RCTs of complex nursing interventions from America, Australia and Europe, published over a two-year period, found that none of the reports gave sufficient detail about the intervention to allow them to be replicated.8 When recruiting patients to a study it is important to attain a sample that closely resembles the patient population from which they are drawn. However, all clinical trials have specific inclusion and exclusion criteria. The rationale for this is that individuals must be appropriate for the study and that the impact of the intervention should not be masked or biased by participants having other important attributes that may confound the effects. Consequently, it has been found that studies tend to recruit people who are not too young and not too old, are English speaking, and preferably have no other health care problems. Following an analysis of research on self-management in type 2 diabetes, it was found that people who were most at risk of complications were also most unlikely to be attracted to or remain in the experimental programme.13 Furthermore, samples tend to: comprise well-educated individuals (or at least literate); include those with the desire and ability to participate in the intervention (whatever that might require); and under-represent ethnic minorities. Therefore, RCTs may be conducted on a sample of people who are not typical of the general patient population. Randomisation is, according to the CONSORT statement, ‘the process of assigning participants to groups such that each participant has a known and usually an equal chance of being assigned to a given group’.14 Randomisation is one of the essential criteria in an RCT, yet – as highlighted by Snowdon et al. – it is known that randomisation can be a difficult concept for potential participants to understand.15 The actual nature of the trial is often poorly understood; the random basis of allocation and its rationale are often problematic. Featherstone and Donovan16 were also interested in patients' perspectives of participation in an RCT. They interviewed 22 men who had been involved in a large RCT and found that most struggled to make sense of aspects of the trial. The process of randomising is also important for those running the trial. In the past it was acceptable for the researchers involved in the study to conduct the randomisation. However, it was realised that randomisation could be fraught with bias created by those running the trial. According to Schulz: ‘Researchers need to realise that, given the opportunity, trial implementers will frequently subvert the intended aims of random assignment.’17 Thus, it was recommended that steps be taken to ensure that those with an interest in the outcomes of the clinical trial cannot influence the allocation of participants to the experimental or control groups. It is now generally agreed that the business of randomly assigning patients to the different groups should be done at a distance by a third party. Methods by which this can be acceptably achieved are included in the CONSORT statement.14 However, in conducting several trials with a behavioural component it became clear to me that randomisation may not always be the answer. If patients have a strong preference for one group or another it can be a snag as it may affect outcome. Although allocated to an intervention group, it does not mean that the individual will undertake the specified behaviours as per protocol and this is particularly challenging if not allocated to the group of their choice (if they had one). Patients asked to self-monitor blood glucose twice a day for four days per week (one fasting and one after main meal). Results to be recorded in a diary. Training in meter use by the DSN – technique observed. Advised on interpretation of results: responses to food/exercise/ sick day rules. Advised to contact DSN if fasting readings five out of seven days were >10mmol/L or post-prandial >12mmol/L. Asked to confirm symptoms of hypoglycaemia with a blood glucose result prior to treatment if possible. Conversely, those in the control group were asked not to monitor for the duration of the trial which was to last for one year. However, at the end of the study it was explained that those in the intervention group could stop monitoring, if desired, while those in the control group would be shown how to monitor if they wished to do so. Although all was explained by the DSNs who knew the patients, there were still occasions when a patient found they were not in their preferred group and asked if they could withdraw from the study as they actually only wanted to take part if they could be in the other group. Naturally, the patients were free to withdraw but the experience served to illustrate that, even though we thought we were explaining the nature of the study carefully, still there were those who did not grasp the basic concept. The need for and implications of allowing for patient preference in clinical trials have been debated.2 On the one hand the implications of employing RCT methodology for all research questions can be questioned, while on the other is the view that only a fully randomised trial is sufficiently robust. While views vary regarding the design and effects of acknowledging patient preference, some of the debate relates to the introduction of new treatments. In complex trials involving behaviour the situation is more complicated. In practice, clinical judgement would be used and care individualised. While not advocating that we only practise according to clinical judgement, there are some interventions that are so dependent on patient motivation and cooperation to undertake behaviour change that it seems unwise to take no account of preference and to allocate such patients to a group they would not choose to join. For example, when designing a study to investigate ‘The effect of an exercise support group on risk factors for renal disease in a diabetic population’, it was agreed that patients meeting inclusion criteria could self-select to either the exercise group or the control group (usual care) as it seemed ridiculous for patients who were willing to participate in the study, but who did not wish to take exercise, to be allocated into the exercise group. However, when the project was submitted for ethical approval, the that, for the of and on the study randomisation into study groups would be and and detail were regarding the for not randomising study Although seemed to the and the research in a situation patient motivation would a large it was the ethical In trials a is used for those people not in the intervention as the can be to like the trial the patient the know who is in the groups. However, the situation is different in behavioural research as it is unlikely that the intervention can be or that the researchers can remain of the group to which the participants et illustrated how individuals can be influenced the of a trial when they the from a All the the – one group of participants were the of the was while those in the other group were they cost were given to the of the to each and their of on a It was found that was in the group who thought they were the more provides an of the of the et al. In a clinical trial the control group is often to The of care in the context of a trial is exactly is it In a trial does care and, if does it it open to bias by the clinical who do not their care to The of the design of this group is illustrated when study results that changes have taken in both groups. If the intervention patients to have more time with than those in the control group, is it the time rather than the actual intervention that the Dr this in an ‘What does make the in to RCTs on patient education. that: when and health care professionals more time with patients in a to them their diabetes, we The design of the control group needs to be thought through to ensure that it does provide an appropriate with the intervention group. Six areas in which are to and which need to be taken into account when designing RCTs for complex interventions, have been This is not a issue and in of the in complex interventions the UK Medical a framework for the design and of complex interventions the described in the Medical ‘A Framework for and of RCTs for Complex to with from the Medical It is that it would be too and to conduct a RCT a of studies up to this According to the and trial work should on a RCT which would then be by Following this framework would enable some of the issues as ‘snags’ to be research have been in a to work through this the research of and into patient education is in this they do that: ‘The time for the evidence from the to after was about This time that the design issues, when complex interventions, need a and that the to be should not be if results are to be robust. The of my presentation was to consider the design of RCTs and how it can have an effect upon the external validity of the study. A study has external validity if the results can be to other cannot be that an experiment can be at least a time and have been does not care about the results of a study its can be used to In order to the necessary levels of control in the experiment, RCTs tend to have validity at the of external are several to external but and intervention support can as when inclusion and exclusion criteria are as they other However, if too many variables are they may a sample that does not the population from which it is and the if found to be may not be of to the population. the nature of the intervention and the of support it are also very important. If the intervention it may be possible to apply these in a research setting while or support have been but such interventions are to be to general health If the experiment is conducted in a the RCT may not and this may be particularly if they are only conducted in of The of RCTs for interventions – such as diabetes management – needs to be in the of the challenges of designing a robust study that can overcome, or at least allow such as have been outlined When evidence created by RCTs for complex interventions be of design issues that may mean the has not been achieved. When published evidence to the care for a patient the research to the population from which the research sample was and the results to this If an RCT – be to be For complex interventions, we need to an open about other robust of are no of are available at