Commentary: Estimating direct and indirect effects—fallible in theory, but in the real world?

A classic finding of the Whitehall Study was that only a third of the association of occupational grade (a socioeconomic ranking of occupations within the British civil service) with coronary heart disease mortality was ‘explained’ after adjusting for known cardiovascular risk factors.1 In current terminology, a third of the association of socioeconomic position with coronary heart disease mortality was estimated as ‘indirect’ via known risk factors, and two-thirds (that unexplained) was estimated as ‘direct’. This direct effect, it is assumed (e.g. ref. 2), represents the mediating or indirect effects of other factors (e.g. unmeasured and/or unknown dietary and lifestyle behaviours, psychosocial factors). For example, subsequent research on the Whitehall Study has suggested that workplace characteristics of control and demand explained much of the remaining two-thirds3—although this analysis has been criticized for conflating measures of socioeconomic position.4,5 The point here, though, is that this original finding from the Whitehall Study is just one example of a widespread (almost universal) practice within epidemiology that aims to describe and quantify causal pathways by controlling for possible mediating variables using standard epidemiological methods. Enter Cole and Hernán6 who, in this edition of the International Journal of Epidemiology, build on prior work of Robins and Greenland (1992)7 and Poole and Kaufman (2000)8 to demonstrate that such ‘standard’ epidemiological practice may be misleading. In brief, using counterfactual models and causal graphs they demonstrate that if an unknown variable (e.g. genotype) confounds the association of the mediating variable with the outcome, then stratifying the exposure-disease association by the mediating variable may not accurately partition the total effect into its direct and indirect components. Previous methodological work demonstrating this fallibility has used completely hypothetical examples. The Cole and Hernán example starts with actual data on randomized aspirin (exposure) and subsequent myocardial infarction (MI; outcome). However, the distribution of the potential mediating variable (platelet aggregation) and the genotype confounder remain constructed. Their data distribution is consistent with a causal and protective effect of aspirin on MI (relative risk of 0.6) being entirely due to platelet aggregation, yet when they stratify the aspirin-MI association by platelet aggregation the relative risk is unchanged. According to standard epidemiological expectation the stratified relative risk should have been 1.0. What happened? To help understand Cole and Hernán's example, I have rearranged the data to determine RRUM|E—the relative risk of the confounder → high platelet aggregation association, stratified by the randomized aspirin exposure (Table 1). It is striking that 95% of those people ‘exposed’ to the confounder U within the unexposed (E = 0; no aspirin on a randomized intervention) had high platelet aggregation compared to 50% among the exposed (E = 1). The difference in the percentages among those not ‘exposed’ to the confounder U is also striking: 50% compared to 5%. Thus, the confounder U is strongly associated with the intermediary variable M. There is also a strong association of the confounder U with D (Table 2). The crude RRUD is 0.23, and stratified by the exposure is either 0.20 or 0.28. But what is more, the presence of the confounder (U = 1) is now protective against disease, despite also being a cause of high platelet aggregation (M = 1) which is positively associated with disease. To summarize, the Cole and Hernán example is equivalent to saying that: aspirin use (E = 1) is strongly protective against MI (D = 1) in truth, the association with aspirin use is purely mediated by low platelet aggregation (M = 1) but as observed, RRED|M was no different from the crude RRED because: actually, some unmeasured confounder was both: strongly associated with high platelet aggregation (which in turn is positively associated with an increased risk of MI) yet, in crude analyses this unmeasured confounding factor was strongly and negatively associated a decreased risk of MI. This type of situation described by Cole and Hernán is extreme—purposely so for illustrative purposes—but nevertheless it is possible. For example, smoking is associated with MI, but smoking may also result in a decreased body mass index that (of itself) is associated with a decreased risk of MI. The key issues are in what direction and with what magnitude does this bias tend to occur in the real world. Regarding direction, we would usually expect the crude association of an unmeasured confounder with an outcome to at least be in the same direction as that mediated by some pathway variable. For example, a ‘bad’ lifestyle (U) is associated with the constellation of high blood pressure, smoking, and being overweight. Consequently, controlling for just blood pressure (M) in the association between, say, socioeconomic status (E) and coronary heart disease (D) will probably overestimate the indirect effect via blood pressure alone as blood pressure is positively correlated with these other risk factors. (This overestimation of indirect effects is opposite to Cole and Hernán's example where the indirect effect was [completely] underestimated.) At least as important as the source of error described by Cole and Hernán is measurement error of potential mediating variables that will usually cause indirect effects to be underestimated. For example, it is highly probably that if multiple (including across the life-course) and more accurate measurements of the known cardiovascular disease risk factors had been available in the Whitehall study,1 more than a third of the association of socioeconomic position with coronary heart disease would have been explained (e.g. ref. 9). There is an urgent need for further methodological research that determines the likely magnitude and direction of bias in the estimation of direct and indirect effects, both by the use of real data sets and sensitivity analyses that determine if less extreme hypothetical examples than that presented by Cole and Hernán still produce noteworthy bias. (Ideally, this methodological research should also incorporate bias from measurement error.) In the meantime, it would be foolish to ignore the alarming findings of researchers such as Cole and Hernán that seriously question the validity of a common epidemiological practice, but neither should we abandon (yet) our standard methods of estimating direct and indirect effects. Accordingly, the three recommendations issued by Cole and Hernán are sensible: plan to collect information on potential confounders of the mediator-outcome association, include these potential confounders in the analysis, and clearly state the implicit assumptions of the standard method used to measure direct and indirect effects. RRUM|E (confounder → high platelet aggregation, given randomization by aspirin exposure) based on Table 3 of Cole and Hernán6 RRUM|E (confounder → high platelet aggregation, given randomization by aspirin exposure) based on Table 3 of Cole and Hernán6 RRUD|E (confounder → disease, given randomization by aspirin exposure) based on Table 3 of Cole and Hernán6 RRUD|E (confounder → disease, given randomization by aspirin exposure) based on Table 3 of Cole and Hernán6 Tony Blakely is funded by the Health Research Council of New Zealand. Useful comments on drafts of this paper were received from Charlie Poole, Stephen Cole and Miguel Hernán.