The Effect of the Heterogeneity Variance Estimator on Some Tests of Treatment Efficacy

The effect of two noniterative heterogeneity variance estimators on two test procedures for the overall treatment effect using the log odds ratio as the outcome measure is investigated. Using simulated significance levels, it is shown that in addition to being sensitive to changes in the magnitude of the heterogeneity variance, the standard test procedure in meta-analysis (or multicenter clinical trials) is also sensitive to the heterogeneity variance estimator used and attains significance levels that are far from the prescribed level. A second test procedure by Hartung [Hartung, J. (1999). An alternative method for meta-analysis. Biometrical J. 41:901-916], is shown to exhibit good control of type I error probability regardless of changes in the magnitude of the heterogeneity variance and the heterogeneity variance estimator used.