Resampling Tests for Meta-Analysis of Ecological Data

Meta-analysis is a statistical technique that allows one to combine the results from multiple studies to glean inferences on the overall importance of various phenomena. This method can prove to be more informative than common “vote counting,” in which the number of significant results is compared to the number with nonsignificant results to determine whether the phenomenon of interest is globally important. While the use of meta-analysis is widespread in medicine and the social sciences, only recently has it been applied to ecological questions. We compared the results of parametric confidence limits and homogeneity statistics commonly obtained through meta-analysis to those obtained from resampling methods to ascertain the robustness of standard meta-analytic techniques. We found that confidence limits based on bootstrapping methods were wider than standard confidence limits, implying that resampling estimates are more conservative. In addition, we found that significance tests based on homogeneity statistics differed occasionally from results of randomization tests, implying that inferences based solely on chi-square significance tests may lead to erroneous conclusions. We conclude that resampling methods should be incorporated in meta-analysis studies, to ensure proper evaluation of main effects in ecological studies.