Choosing the Correct Unit of Analysis in Medical Care Experiments

The statistical methodology of health research experiments published in Lancet, the New England Journal of Medicine, and Medical Care between 1975 and 1980 for the presence or absence of an error of experimental design and analysis was examined. The error is the result of inappropriately using patient-related observations as the unit of analysis to form conclusions about provider behavior or outcomes determined jointly by patients and providers. The error was present in 20 of 28 (71%) health care experiments addressing an issue of health provider professional performance. Its usual effect is to increase erroneously the power of an experiment to detect differences between experimental and control groups. It is likely that this type of error could be avoided by the explicit and prospective definition of hypotheses and the populations to which they are intended to pertain.