What Impacts Should We Expect From Tutoring at Scale? Exploring Meta-Analytic Generalizability

U.S. public schools engaged in an unprecedented effort to expand tutoring in the wake of the COVID-19 pandemic. Broad-based support for scaling tutoring emerged, in part, because of the large effects on student achievement found in prior meta-analyses. We conduct an expanded meta-analysis of 263 randomized controlled trials and explore how estimates change when we better align our sample with a policy-relevant target of inference: large-scale tutoring programs aiming to improve performance on independent, standardized tests. Pooled effect sizes from studies with stronger target-equivalence are .16 to .22 standard deviations, relative to .40 standard deviations in our full sample. This result is driven by stark declines in pooled effect sizes as program scale increases. We explore four hypotheses for this pattern and document how a bundled package of recommended design features serves to partially inoculate programs from this attenuation at scale.