Breaking the matches in a paired t‐test for community interventions when the number of pairs is small

There is considerable interest in community interventions for health promotion, where the community is the experimental unit. Because such interventions are expensive, the number of experimental units (communities) is usually small. Because of the small number of communities involved, investigators often match treatment and control communities on demographic variables before randomization to minimize the possibility of a bad split. Unfortunately, matching has been shown to decrease the power of the design when the number of pairs is small, unless the matching variable is very highly correlated with the outcome variable (in this case, with change in the health behaviour). We used computer simulation to examine the performance of an approach in which we matched communities but performed an unmatched analysis. If the appropriate matching variables are unknown, and there are fewer than ten pairs, an unmatched design and analysis has the most power. If, however, one prefers a matched design, then for N < 10, power can be increased by performing an unmatched analysis of the matched data. We also discuss a variant of this procedure, in which an unmatched analysis is performed only if the matching 'did not work'.