Meta-Analysis of Problem-Solving Studies: A Critical Response

In the past decade, meta-analysis has been applied with increasing frequency to the synthesis of educational research findings. My initial skepticism of its use has. if anything. grown greater during this period. Some of the difficulties with the methods described in the early texts (Glass. McGaw, & Smith, 1981; Hunter, Schmidt, & Jackson, 1982) were, I would begin by acknowledging. effectively addressed through careful delineation of the assumptions underlying the statistical procedures and greater attention to the limits of their applicability (Hedges & Olkin, 1985). Meta-analysis can in fact provide answers to certain kinds of carefully formulated questions, when great caution is exercised (e.g., see Wachter, 1988, and Iyengar, 199 1, for interesting and critical perspectives). Nevertheless, a substantial fraction of meta-analytic studies in education violate the conditions that would be necessary for acceptance of their conclus ions. When this occurs, the method lends an aura of quantitative certainty to questionable generalizations.