The implications of variation in outcome between health professionals for the design and analysis of randomized controlled trials

Methodological work on randomized trials has largely concerned pharmacological interventions in which the effects of the attending health professional may be regarded as minor. In other clinical settings, such as surgery, talk or physical therapies, staff specific variation may make generalization problematic, undermining the value of the trial. Such variation has been the basis of some objections to controlled trial methodology and non-acceptance of trial results. The implication of this source of variation will be considered for studies in which different types of health professional deliver the intervention in each arm of the trial. Such a trial may involve individual patient or group randomization. Whichever method is used, it is argued that variation in outcome between health professionals may lead to design effects. These issues will be illustrated using data from a large trial comparing primary care service delivered by two types of medical doctor. Random effect models are most suitable for analyzing this type of trial, as they allow adjustment for patient characteristics whilst controlling for design effects. This type of model illustrates that there can be substantial variation in the performance within each category of doctor.