Testing the Equality of Two Independent Binomial Proportions

Abstract The problem of testing equality of two independent binomial proportions is reexamined. A recent article by D'Agostino, Chase, and Belanger (1988) argues that the Fisher exact test and the Yates chi-squared test with continuity correction are much too conservative in small samples. The authors proposed using a studentized version of the Pearson chi-squared test, and showed that the empirical size of this test is generally close to the nominal level in repeated product-binomial sampling. Although the article is persuasive on its own terms, two central issues are completely ignored: (a) the propriety of analyzing tests based on discrete data using fixed nominal levels of significance, and (b) the question of whether the empirical size should be computed in repeated samples that fix one or both of the margins. The latter is the key issue; Yates and others (1984) argued that both margins should be held fixed for inference, even though only one margin is fixed by the product-binomial sampling design. The arguments for conditioning on both margins are reviewed and found persuasive; however, it can also be argued that acceptance of the idea of conditioning leads to the likelihood principle, and hence the rejection of significance tests as valid measures of inferential evidence.