Advanced Methods for Evaluating the Impact of Family Planning Communication Programs: Evidence from Tanzania and Nepal

Evaluations of national family planning communication campaigns are often complicated by nonrandomized research designs and self-reported measures of exposure to media messages, both of which can bias estimates of campaign effectiveness if suitable analytical methods are not employed. This study uses three methods to address these problems: (1) single-equation multiple regression analysis; (2) propensity score matching (PSM); and (3) maximum-likelihood simultaneous equation models (SEM), namely bivariate probit. We use these methods to evaluate the impact of family planning health-communication campaigns on the use of modern family planning methods, focusing primarily on the radio drama Zinduka! in Tanzania and Ghanti Heri Haad Nilaun in Nepal. We find that these methods led to substantially different estimates of the magnitude of the impact of these communication campaigns. Specifically, we found that self-reported exposure to each radio program was exogenously related to family planning use and that PSM and multiple regression analysis gave similar estimates in those situations. An indicator of exposure to any type of family planning message was found to be endogenous, in which case bivariate probit and PSM gave different estimates of program impact. Researchers should, therefore, pay close attention to the results of tests for endogeneity and identification that guide the appropriate use of the different methods.