Arguing for Covariate Balance

Covariate balance is an important component of causal identification in observational studies. Balance assessment is common empirical practice, but typical implementations erroneously reflect the a priori expectation that balance is achieved, as is plausible in a randomized experiment. Consequently, analysts do not provide sufficient evidence or compelling argumentation supporting balance. Instead, standard practice involves checking for patterns of statistical significance consistent with a randomized treatment. I propose an alternative framework that facilitates positive, persuasive covariate balance tests with observational data. The framework evaluates multiple moments of the covariate distributions and estimates balance statistic uncertainty using either classical or Bayesian bootstrapping. I demonstrate that it is more informative and intuitive compared to the conventional emphasis on the statistical significance of covariate mean differences. The framework greatly enhances insight into the credibility of observational designs by encouraging analysts to argue positively for covariate balance rather than tenuously suggest the absence of imbalance.