Ten reasons to conduct a randomized study in quality improvement

In the years 2004–06, the Journal has published 192 papers (excluding editorials and letters) of which eight (4.2%) were randomized experiments [1–8]. Four per cent is not enough, considering that randomization is the most direct path to a causal inference. To be effective, we need to understand what causes what in quality improvement. This aim is not compatible with weak research methods. Some quality experts claim that the evaluation of quality projects requires less scientific rigour than ‘academic’ research. It their view, small uncontrolled before/after studies or even spontaneous experiential learning is sufficient. How could that be possible? Quality improvement deals with complex systems, multifactorial interventions, unpredictable individuals, and multifaceted outcomes. If the object of inquiry is complex, the research methods must be clever and imaginative, but also rigorous. But I digress. The bottom line is: we need more randomized studies in the field of quality improvement. Here are some reasons why you should consider performing one. People have the notion that randomized studies are complicated, whereas observational studies are nice and simple. It is the other way around. Whatever intervention you allocate randomly will reveal its effect, or lack thereof, without artefacts (which, granted, may not be the desired outcome on occasion). By contrast, the interpretation of an observational study will require a lot of thinking about possible confounders and a long ‘limitations’ section when you write up the findings. Because you will not have to worry about confounding variables, you will not have to measure them. There is a necessary trade-off between the quantity and the quality of information that can be collected in any study. In a randomized trial, you can focus on a short list of key variables, measure them with all the necessary precision, and still have a manageable dataset. Another beauty of a randomized experiment is that the main analysis is extremely simple: it is a head-to-head comparison of two (or more) groups. If you can do a chi-squared test on a 2 × 2 table, or run a t-test, you will be fine most of the time (to be honest, true statistical expertise will be necessary at times, as in the case of cluster randomization, repeated measurements, many missing observations, etc.). The paradigm of a randomized clinical trial, the pharmaceutical phase III trial, usually examines a simplistic intervention—an antibiotic for an infection, an antidepressant for depression, etc. However, the randomized design works just as well with complex interventions. Diagnostic strategies, strategies of diffusion of innovations, and public health programmes—for any of these, the key question is ‘does it work better than usual care’, and all can be tested in randomized trials. You may not be able to say what component of a quality improvement intervention was the most critical, but that is not the most relevant question in real life. These ‘pragmatic’ randomized trials are much underused [9,10]. Most pragmatic trials do not require placebos or blinding. The placebo effect is part and parcel of any intervention that aims for systemic change. People have to be motivated to work differently. Hiding the fact that change is underway does not make sense. Another reason for having a placebo is to allow blinding, i.e. an unbiased assessment of effects. There are other ways of achieving this aim: reliance on validated instruments, separation of the roles of assessor and promoter of the intervention, or use of routinely collected outcome data. We usually think of randomizing patients, but this was done in only one of the eight trials published in this journal [4]. Random allocation of health care providers—doctors, hospitals, etc.—is often the appropriate approach for quality improvement interventions [3,5,6]. Alternatively, within an organization, time-periods can be randomly allocated to an active or a control intervention, in the manner of N-of-1 clinical trials. Moreover, randomized experiments can be performed to compare feedback methods for policy makers [1], perceptions of medical errors by the public [2], survey methods [8], or the usability of medical devices [7]. Because an observational study does not interfere with practice, observational research makes things a bit too easy for the researcher and hence encourages sloppiness. Measure a lot of variables, cross-tabulate them back and forth, and eventually something will pop up that will be interesting. By contrast, when you randomize, you have to think hard about what the key scientific question is—you only get that one chance. And you have to commit to that research question in writing. But the hard thinking at the start of the project pays off eventually in the relevance of the results. It is unusual to perform a randomized study and not be able to publish the results rapidly and in a reputable journal. You get instant credibility with editors and reviewers by randomizing. No reviewer of your manuscript will be able to write in all impunity: ‘this association may be due to unmeasured confounders’—the kiss of death of an observational study, because by definition what you have not measured you cannot adjust for. The Reizenstein prize rewards the best paper published in the International Journal for Quality in Health Care of the previous year. It is awarded based on the votes of the members of the journal’s editorial board. Two of the last three laureate papers were based on a randomized design [2,5]. Thus, the risk of winning the prize is 25% (2 of 8) if your study is randomized, and 0.5% (1 of 184) if it is not—a relative risk of 50. But then again, we did not randomly allocate these studies to be randomized or not.... Performing your first randomized trial will be exhilarating, like the first time you could ride a bicycle. Take off these training wheels!