A Comment on “Raising Health Awareness in Rural Communities: A Randomized Experiment in Bangladesh and India” by Siddique et al. (2024)

Siddique et al. (2024) report large effects of a mobile phone-based health awareness campaign in a field experiment conducted in rural Bangladesh and India during the COVID-19 pandemic. Treatments are stated as being randomized at the village level in the Bangladesh arm, but analyses of available replication materials from this and related field experiments show that this was not the case: participants retained their treatment group assignment from two earlier studies run by the same NGO. Because these earlier studies reported large and persistent treatment effects, the treatment groups in Siddique et al. (2024) could plausibly differ systematically, raising issues of confounding and interaction effects that invalidate the analyses and interpretations of this study, affecting both internal and external validity.