Why the Average Treatment Effect for the Overlap Population (ATO) Matters for Natural Experiments

Natural experiments—defined as research designs in which treatment is plausibly randomly assigned by a mechanism outside the researcher’s control—are increasingly common in empirical legal studies. Such designs can greatly strengthen the credibility of causal inference. However, the fact that treatment assignment is outside the control of the researcher also creates at least two distinct challenges for credible causal inference. First, lack of researcher control over treatment assignment can strain the overlap assumption required for many identification strategies. Second, the interventions implied by standard estimands are often infeasible in these settings. In this paper, we discuss a relatively new causal estimand that we argue helps mitigate both concerns: the average treatment effect for the overlap population (ATO) ( Li et al., 2018 ). ATO focuses attention on regions of the covariate space where overlap between treated and control units is strongest. It also has an interpretation directly relevant to situations in which one seeks to understand the effects of treatments assigned by an exogenous mechanism outside the researcher’s control. We illustrate these ideas by reanalyzing a study of ethnic bias in judicial decision making in Kenya ( Choi et al., 2022 ).