What question did this study set out to answer?

The primary aim is to explore methods for evaluating the causal impact of policy changes using machine learning.

August 7, 2015

Machine Learning and Causal Inference for Policy Evaluation

Key Points

The primary aim is to explore methods for evaluating the causal impact of policy changes using machine learning.
Review of existing literature on causal inference in various fields.
Evaluation of techniques like propensity score weighting and regression analysis to estimate treatment effects.
Discussion of structural modeling and its application in understanding policy effects.
Identification of assumptions that allow conclusions similar to randomized experiments.
Successful integration of machine learning with traditional causal inference methods for better policy evaluation.
Highlighting the shift from prediction to causal understanding in supervised machine learning.

Abstract

A large literature on causal inference in statistics, econometrics, biostatistics, and epidemiology (see, e.g., Imbens and Rubin 2015 for a recent survey) has focused on methods for statistical estimation and inference in a setting where the researcher wishes to answer a question about the (counterfactual) impact of a change in a policy, or "treatment" in the terminology of the literature. The policy change has not necessarily been observed before, or may have been observed only for a subset of the population; examples include a change in minimum wage law or a change in a firm's price. The goal is then to estimate the impact of small set of "treatments" using data from randomized experiments or, more commonly, "observational" studies (that is, non-experimental data). The literature identifies a variety of assumptions that, when satisfied, allow the researcher to draw the same types of conclusions that would be available from a randomized experiment. To estimate causal effects given non-random assignment of individuals to alternative policies in observational studies, popular techniques include propensity score weighting, matching, and regression analysis; all of these methods adjust for differences in observed attributes of individuals. Another strand of literature in econometrics, referred to as "structural modeling," fully specifies the preferences of actors as well as a behavioral model, and estimates those parameters from data (for applications to auction-based electronic commerce, see Athey and Haile 2007 and Athey and Nekipelov 2012). In both cases, parameter estimates are interpreted as "causal," and they are used to make predictions about the effect of policy changes. In contrast, the supervised machine learning literature has traditionally focused on prediction, providing data-driven approaches to building rich models and relying on cross-validation as a powerful tool for model selection. These methods have been highly successful in practice. This talk will review several recent papers that attempt to bring the tools of supervised machine learning to bear on the problem of policy evaluation, where the papers are connected by three themes.

Mark Helpful

Bookmark

Relay