Covariate Adjustment in Political Science Experiments: Progress and Recommendations

Covariate adjustment can reduce the standard error of experimental treatment effect estimates without increasing the sample size but is vulnerable to abuse unless a specific procedure is preregistered. We assess adjustment practices in ten political science journals in 2013, 2018, and 2023–24. Encouragingly, preregistration and covariate adjustment have each been rapidly adopted. Preregistration is also a good predictor of what practices are used in articles. However, just 12% of articles published in 2023–24 preregister a specific covariate adjustment procedure and use adjustment for the main estimate. Although preregistering an adjusted estimator is not always advisable (when covariates are unavailable or preregistration is not possible), there is substantial room for improvement. We recommend that when feasible, researchers preregister a specific covariate adjustment procedure and use it for their preferred estimate regardless of covariate balance. This may require a shift in preregistration culture away from predicting results and toward constraining analysis.