What question did this study set out to answer?

The aim is to examine the implications of rerandomization on the credibility of randomized field experiments.

March 12, 2026Open Access

What is wrong with rerandomization in randomized field experiments?

Key Points

The aim is to examine the implications of rerandomization on the credibility of randomized field experiments.
Analyzed effects of rerandomization on covariate balance in experimental groups.
Discussed necessity of adjusting p-values and selecting predictive covariates.
Evaluated the impact of increased degrees of freedom on analytical outcomes.
Found that rerandomization reduces imbalances in covariates, enhancing treatment effect estimates.
Highlighted that increased flexibility may lead to p-hacking for significant results.
Concluded that these practices can undermine the credibility of experimental findings.

Abstract

Abstract Rerandomization is a new technique employed in randomized field experiments (RFEs) to enhance the balance of covariates across experimental groups. Rerandomization is recommended because it reduces imbalances in known covariates, thereby enhancing the precision of the average treatment effect estimates. However, employing rerandomization necessitates adjusting observed p-values, using regression-based inference, and selecting predictive covariates. All these amendments increase the number of researcher degrees of freedom, i.e., methodological decisions involved in designing and analyzing experiments. I argue that this increased analytical flexibility may be misused to p-hack for statistically significant or preferred results, thereby reducing the credibility of the results.

Mark Helpful

Bookmark

Relay

View Full Paper