In Pursuit of Evidence in Air Pollution Epidemiology: The Role of Causally Driven Data Science

The impact of air pollution on health gained widespread attention in 1952 when pollution from coal burning in London, with smoke so thick that drivers needed headlights at all times of day, was linked to a dramatic rise in the number of deaths.1,2 Over the past decades, evidence of adverse population health effects has accumulated from many thousands of epidemiologic studies, suggesting this is a critically important public health problem. While at the individual level the relative effects of exposure tend to be small, and potentially large confounding biases are difficult to rule out, air pollution is ubiquitous and the exposed population is enormous. Thus, these adverse effects have important policy implications and intense scrutiny of these epidemiologic findings is inevitable.3,4 Recently, the emergence onto the statistics landscape of modern causal inference methods, as well as machine learning and other novel data science techniques, has generated excitement and a sense of promise that estimation of the causal effects of air pollution exposures on health outcomes could be more definitively described.5–7 (By causal effects we mean the effects that would be seen under experimental changes of exposures. By causal inference we mean the process of inferring causal effects from data.) We share the excitement of others that the discipline of causal inference has the potential to advance air pollution policy and allow the integration of modern statistical tools into air pollution epidemiology, but we also caution against unrealistic expectations by highlighting important difficulties ahead. Our goal is to provide a glimpse of the opportunities afforded by the use of causal inference and data science methods, raise awareness about some of the outstanding challenges, and inspire others to join on the efforts to overcome them. In our note, we first discuss broad concepts that are relevant across applications, and then focus issues germane to air pollution epidemiology. THERE IS A NEED FOR DELIBERATE CAUSAL INFERENCE IN POLICY-RELEVANT RESEARCH When the goal of a scientific study is to inform policy, describing associations between exposures and outcomes generally does not suffice: an assessment of causal effects is needed.8 Unless a particular study design very clearly allows such an assessment, as in an appropriately designed and conducted randomized trial, epidemiologists and biostatisticians typically word their findings carefully to avoid implying causation.9 Nevertheless, since the premise of these analyses is to bring to light potentially causal relationships, despite cautious wording, investigators and end users both may implicitly infer causation. For this reason, we believe it is important for analysts to employ methodology that allows causal inference under the most realistic set of conditions possible, and to ensure clear communication of these conditions to facilitate the interpretation of the results. CONVENTIONAL MODEL-BASED APPROACHES MAY BE INADEQUATE FOR INFERRING CAUSAL EFFECTS In many disciplines, conventional regression models (e.g., the very many varieties of the linear model) are considered a cornerstone of traditional statistical practice for inference about health effects. Indeed, the great majority of air pollution epidemiology studies estimate health effects as relative risks or hazard ratios obtained using regression models. In the conventional model-based approach, it is customary to begin by selecting a particular regression model, often on the basis of the type of outcome under study (e.g., logistic regression for binary outcomes and proportional hazards model for survival outcomes). How notions of causality may be ascribed is then considered as a second step, often restricted to consideration of which potential confounders ought to be included in the model. In this conventional approach, investigators often focus on parameters most conveniently reported on the basis of the model chosen (e.g., odds ratio in logistic regression and hazard ratio in proportional hazards model). Unfortunately, this common and traditional use of conventional regression models for causal inference has at least two important drawbacks. First, whether the estimation procedure used yields valid inferences on a scientifically meaningful estimand (i.e., the quantity being estimated) – let alone one with a causal interpretation – generally hinges on the assumption that the specified regression model accurately reflects the relationship modeled.10–12 And that the true underlying mechanism follows the simple form of conventional regression models is often a strong assumption without much prior empirical support. Additionally, well-intentioned efforts to reduce model misspecification through an iteration of model revisions guided by diagnostics (e.g., residual plots) can compromise the interpretation and validity of inferences from these models.13–15 Accounting for informal model selection in model-based analyses is difficult and remains an unsettled topic of methodologic research.16–20 Second, even in the ideal scenario in which all relevant confounders are accounted for and the regression model postulated a priori holds true, model-based regression coefficients corresponding to the exposure of interest may still not refer to the causal contrast desired to address the scientific question at hand. This may occur, for example, because the regression coefficients quantify a causal effect on a different scale than desired (e.g., odds ratio vs. relative risk) or do not provide the population-level summary desired (e.g., conditional vs. marginal interpretation). The limitations of conventional model-based causal inference are exacerbated when modeling is performed on scales less amenable to causal comparisons (e.g., the hazard ratio), or when the exposure of interest occurs over a longer period of time.21–25 HOW DO MODERN CAUSAL INFERENCE METHODS DIFFER? Modern causal inference methods help circumvent the limitations of conventional model-based approaches. First, as we discuss below, these methods facilitate the use of more flexible learning approaches. As such, they eliminate the overreliance of conventional model-based approaches on strong model assumptions, thereby increasing the reliability of the resulting scientific findings. Second, in modern causal inference, the causal estimand is rigorously defined before the inferential method is determined. Because it is deliberately chosen rather than inherited from the model choice that the outcome data type might suggest, the estimand is more likely to be directly relevant to address the scientific question. These points are discussed further below. The choice of causal estimand plays a fundamental role as the starting point of all statistical considerations in modern causal inference methods. Counterfactual (or potential) outcomes, which are used to refer to hypothetical versions of the outcome under different exposure profiles of interest, play a key role in this first step. Causal estimands are generally expressed as summaries of the distribution of counterfactuals.26 For example, in a simple binary exposure setting, the average causal effect is the mean value of the difference between the counterfactual outcomes corresponding to each of the two exposure profiles (e.g., exposed at high vs. low levels). These causal estimands cannot be directly estimated because only a single counterfactual can be observed for each study participant – for example, for an individual having been exposed at a high exposure level, only the counterfactual outcome corresponding to high exposure is observed. To identify causal estimands from the observed data, there must exist an appropriate “bridge” (i.e., mapping) from the counterfactual world, where causal estimands are defined, to the observed world, in which data are collected. This bridge must allow the causal estimand to be expressed as a statistical estimand, that is, as a summary of the distribution of the observed data (Figure 1). The existence of a bridge hinges on meeting the causal conditions, many of which are untestable. For the average causal effect, typical causal conditions used for identification include the following: consistency (i.e., the intervention defining exposures is unambiguous); the stable unit treatment value assumption (i.e., no interference); positivity (i.e., each participant could have been observed in the counterfactual exposure group); and (sequential) ignorability (i.e., exchangeability or no unmeasured confounding).26 When such a bridge exists, the causal estimand is said to be identified.26 For example, if the average causal effect (say of a high vs. low binary exposure) is the causal estimand of interest, the destination of one such bridge is the G-computation formula. The latter is given by the population average of the difference in strata-specific mean outcomes among individuals with high vs. low exposure, where strata correspond to subpopulations defined by confounder levels.26 We invite readers to consult the textbook of Hernán and Robins for an in-depth study of foundational ideas in causal inference.26 For a comprehensive discussion of causal inference in air pollution epidemiology, we point readers to the recent review by Bind.7FIGURE 1.: Framework for causally guided data science.Once identification of the causal estimand is established, the resulting statistical parameter can be estimated using a variety of strategies. At this point, the problem is purely statistical: the fact that the statistical parameter corresponds to the causal estimand under certain causal conditions is irrelevant to how estimation and inference should then proceed. Simplifying statistical conditions, such as parametric forms (e.g., linear mean regression model), can be imposed on the observed data distribution to facilitate estimation and inference, but they are not strictly needed. This is where flexible learning techniques, such as machine learning, can be deployed to perform statistical inference less prone to bias due to model misspecification. Prior knowledge, including characteristics of the study design, informs all the conditions and can justify imposing more stringent statistical conditions that can be leveraged to generate more precise inference. Of course, whether or not a causal interpretation may be ascribed to the inferences drawn depends on the validity of the causal conditions. In summary, as we highlight in the figure, a desirable feature of the causal inference framework is that it separates conditions as being either causal or statistical, with the latter generally being imposed out of convenience rather than necessity. In contrast, in the conventional model-based approach, causal and statistical conditions are often entangled. As such, without additional work, it is often difficult to determine whether identification is in fact largely driven by strong statistical modeling conditions (e.g., assumption of linearity of the mean outcome beyond the range of observed exposures27) – this reduces transparency and rigor in the conduct of causal inference. Explicitly disentangling causal and statistical conditions is also important because very different strategies exist to relax each set of conditions. CAUSALLY GUIDED DATA SCIENCE HAS THE POTENTIAL TO ADVANCE POLICY-RELEVANT DECISION-MAKING Because causal estimands, such as the average causal effect, have clearer scientific interpretability than statistical estimands based on (plausibly misspecified) conventional models (e.g., coefficients in regression models), they are better suited to inform air pollution policy-making.28 These causal estimands can quantify the health consequences of hypothetical (possibly regulatory-driven) changes in exposure, and can predict the effects of future interventions or policies.5 Modern causal inference methods therefore widen the scope and increase the granularity of policy-relevant scientific questions that can be addressed from the available data.5 Additionally, reliable inference in air pollution epidemiology is needed, particularly in view of the wide-ranging implications of air pollution policies. Results from analyses that build upon unrealistic conditions may be misleading. Causal conditions allow identification of the causal parameter without generally implying constraints on the observed data distribution. As discussed above, the use of conventional statistical models, such as conventional parametric models, often impose unnecessary conditions for the validity of inferences, regardless of whether the causal conditions are met. This is where we believe developments in data science can have the greatest impact. Statisticians and other data scientists have developed many flexible learning algorithms to reliably and robustly uncover structure from data. Their use, however, does not imply that more conventional learning methods (e.g., linear regression) must be discarded. Certain ensemble learning methods (e.g., the Super Learner, an example of model stacking) are able to combine algorithms in any user-specified collection of learning strategies in an automatic, data-driven, and optimal fashion.29,30 This eliminates the need to bet on any particular learning tool and also largely negates the relevance of model diagnostics. To perform optimal inference for the statistical parameter of interest that identifies the causal estimand, the output of any flexible learning method must generally be corrected to achieve an optimal bias-variance trade-off for the desired estimand. There are several possible strategies for doing so, some of which have been used for decades (e.g., estimating equations methodology) and others are more recent innovations (e.g., targeted minimum loss-based estimation).31–33 What emerges is thus a two-step process, wherein relevant features of the observed data distribution are estimated flexibly and then corrected to ensure valid inference for the actual target of inference. The study of the tools and relevant issues to consider in this process has been the focus of targeted learning, a rapidly expanding area of methodologic research.32,34–39 These tools allow practitioners to employ flexible learning strategies, thus facilitating robust analyses based on as few statistical conditions as possible. They also enable reliable uncertainty assessment, including confidence intervals and P-values, a critical ingredient in the interpretation of results for the sake of policy-making. When causal conditions fail to hold (e.g., there is interference or unmeasured confounding), a bridge may not exist, and full identification of the causal estimand is then not possible. In some cases, partial identification, wherein bounds on the true causal estimand are identified, holds.40 Full identification can be recovered by modifying the definition of the exposure or the target population. An alternative strategy is to find an instrument – a cause of exposure that otherwise has no bearing on the outcome – to help circumvent failure to account for all important confounders, including the ones that are unmeasured.41,42 Examples include quasi-experiments such as the coal ban in Dublin and the traffic plan implemented during the Beijing Olympics.43,44 Regardless, even when causal conditions do not strictly hold, causally-motivated estimands still have a transparent, model-agnostic interpretation that is often more sensible than that of conventional model-based estimands. We believe this enables more informed policy-relevant decision making about air pollution. CAUSAL INFERENCE IN AIR POLLUTION EPIDEMIOLOGY IS INHERENTLY CHALLENGING Investigating the causal effect of exposure to air pollution on health outcomes is an intrinsically challenging endeavor.45,46 We highlight only a few of these challenges below. First, obvious ethical considerations often preclude the conduct of randomized trials, a design that allows easy derivation of causal inferences. Observational studies face confounding biases that may be difficult to fully account for. Furthermore, these biases can be in either direction and can have unknown magnitude. This challenge is magnified because for individuals the relative increase in risk from air pollution exposures is typically small.47 Of course, this is not to say that these adverse effects should be overlooked – ubiquitous air pollution exposures can still affect individuals, and, more importantly, accumulate over large populations, resulting in a substantial burden for communities and health systems.48 Uncontrolled confounding results in violation of ignorability, a key causal condition, and is difficult to avoid air pollution epidemiology. Second, when the health effects arise from long-term exposures, defining causal effects of interest can be challenging because of the many possible definitions of counterfactual outcomes. Defining the counterfactual outcomes requires selection of the relevant interventions on exposures. What exposure duration should be considered? Should daily exposure be fixed at a particular level across time, or allowed to vary within a window? Being more or less strict in defining patterns of exposure leads to a trade-off between the interpretability of causal effects and the feasibility of the statistical problem. Oversimplification of the complex exposure process – for example, by ignoring the time-varying nature of exposures – can potentially invalidate causal inferences, as some authors have recently argued.49 Marginal structural models, a set of tools for performing parsimonious causal regression, and causal estimands based on stochastic interventions seem particularly promising to tackle these challenges.25,50–55 Third, even when using causal inference methods, disentangling the causal effects of exposure to a particular pollutant may be difficult if it tends to co-occur with other pollutants.56–63 This may be especially true if these pollutants emanate from the same source. Since the co-occurring exposures possibly confound the relationship between the pollutant and health outcome of interest, and must therefore be accounted for, positivity violations are likely. Learning accurately how the outcome differs based jointly on the various exposures and confounders is then difficult, since the health effects are at best only weakly identified. Fourth, accurately measuring exposure to air pollutants is itself a challenging task.47,64 While technical advances have made it possible to outfit individuals with personal sensors capable of measuring exposure histories, implementing this in large-scale studies is prohibitively costly. Instead, by combining data obtained from regulatory monitoring stations, geographic information systems, and satellite imaging, researchers have developed historical exposure models used to predict pollutant levels in space and time, particularly at unmonitored locations.65–69 However sophisticated these models may be, their Accounting for their uncertainty and biases largely remains an problem in air pollution epidemiology, recent advances have been Additionally, even ignoring the of historical pollutant individual would be needed to exposure most studies use pollutant levels at as a to exposure ignoring the fact that most exposures may in an different is important to the of these issues on causal inference and to identify possible causal methods are and their remains the focus of advances in and tools are likely needed before many practitioners be to as include the following: appropriate out relevant causal identification (i.e., these causal estimands to summaries of the distribution – the and inferential that data science tools (e.g., machine These substantial in recent technical such as causal inference, targeted learning, and machine learning, which many do not in The of these modern tools to many scientists may also be a to possibly researchers to in the use of more conventional methods. BE CAUSAL INFERENCE We view the of data science tools and causal inference into air pollution epidemiology as an important As and methods for causal inference can play a this is because they include a more of all the that are needed to for confounding with regression modeling and therefore can infer causality in analyses of Nevertheless, we cannot the emergence of a methodologic since many of the challenges of valid causal inferences about air pollution health effects from features of the nature of the available data. In other no statistical is likely to overcome these challenges methods must be with the use of study or data to the particular difficulties in air pollution epidemiology. scientists have because epidemiologic studies provide of and may not accurately predict the of air they should not be thereby a large of evidence over the despite the of air can be set for an of In our causal inference methods should not be used as to science against cannot for the data, study and tools that ensure causal evidence is and does not public This was to the discussion at the in and at the The authors to and for on of this work, and for their THE is an of at the of of and an in the and at the at the of and causal inference, survival and statistical epidemiology. on and robust methods for performing statistical inference with machine learning is the is a of at the of and of the at on and methods for the of data to identify and the health of and inform is a of the of is a of and and health at the of of focus on statistical methods for the health effects of and they include study design, exposure modeling and and estimation of exposure effects with to a range of health outcomes. is a of the and on the