Econometric analysis typically focuses on the statistical properties of fixed estimators and ignores researcher choices. In this article, I instead approach the analysis of experimental data as a mechanism‐design problem that acknowledges that researchers choose between estimators, sometimes based on the data and often according to their own preferences. Specifically, I focus on covariate adjustments, which can increase the precision of a treatment‐effect estimate, but open the door to bias when researchers engage in specification searches. First, I establish that unbiasedness as a requirement on the estimation of the average treatment effect can align researchers' preferences with the minimization of the mean‐squared error relative to the truth, and that fixing the bias can yield an optimal restriction in a minimax sense. Second, I provide a constructive characterization of treatment‐effect estimators with fixed bias as sample‐splitting procedures. Third, I discuss the implementation of second‐best estimators that leave room for beneficial specification searches.
Jann Spiess (Wed,) studied this question.