Does Evaluation Design Impact Communication Campaign Effect Size? A Meta-Analysis

This is the authors' abstract. We don't add key points for this paper.

To examine how methodological choices and artifacts of evaluations influence the estimated effect size of media campaigns on health behavior, we meta-analyzed 44 mediated health campaigns. Effect sizes were largest when the posttest evaluation occurred close to the end of the campaign and utilized a one-group partition-by-exposure design. Use of a self-selected sample and greater sample attrition biased the results by inflating effect sizes. Improvements in the control community over time made it more difficult to detect intervention effect sizes. Design factors that did not result in different effect sizes included panel versus cross-sectional, behavioral versus physiological measure, and nonequivalence in the control and intervention communities at baseline. The results may be useful for evaluators and researchers who seek to understand the impact of methodological factors on the interpretation of campaign effects.