Factors affecting power of tests for multiple binary outcomes

Frequently in clinical studies a primary outcome is formulated from a vector of binary events. Several methods exist to assess treatment effects on multiple correlated binary outcomes, including comparing groups on the occurrence of at least one among the outcomes ('collapsed composite'), on the count of outcomes observed per subject, on individual outcomes adjusting for multiplicity, or with multivariate tests postulating either common or distinct effects across outcomes. We focus on a 1-df distinct effects test in which the estimated outcome-specific treatment effects from a GEE model are simply averaged, and compare it with other methods on clinical and statistical grounds. Using a flexible method to simulate multivariate binary data, we show that the relative efficiencies of the assessed tests depend in a complex way on the magnitudes and variabilities of component incidences and treatment effects, as well as correlations among component events. While other tests are easily 'driven' by high-frequency components, the average effect GEE test is not, since it averages the log odds ratios unweighted by the component frequencies. Thus, the average effect test is relatively more powerful than other tests when lower frequency components have stronger associations with a treatment or other predictor, but less powerful when higher frequency components are more strongly associated. In studies when relative effects are at least as important as absolute effects, or when lower frequency components are clinically most important, this test may be preferred. Two clinical trials are discussed and analyzed, and recommendations for practice are made.