Exact and asymptotic tests for homogeneity in several 2×2 tables

This paper presents the results of a Monte Carlo study comparing the performance, in terms of size and power, of six exact and six asymptotic tests for the homogeneity of odds ratios in several 2 x 2 contingency tables. With a small sample size or sparse data structure, the exact tests performed better than the asymptotic tests because they maintained the nominal size and, in some situations, had slightly higher power. Among the exact tests, we recommend the Zelen, Pearson chi-square and scores tests. Among the asymptotic tests, the Breslow-Day and Pearson chi-square tests were slightly better in some situations than the unconditional and conditional score tests. However, both exact and asymptotic tests had low power for small strata sizes, even with moderate to large heterogeneity of odds ratios. Corroborating previous findings, the asymptotic unconditional likelihood ratio test was too liberal in terms of size.