The formal approach to quantitative causal inference in epidemiology: misguided or misrepresented?

Two recent articles, one by Vandenbroucke, Broadbent and Pearce (henceforth VBP)1 and the other by Krieger and Davey Smith (henceforth KDS),2 criticize what these two sets of authors characterize as the mainstream of the modern ‘causal inference’ school in epidemiology. The criticisms made by these authors are severe; VBP label the field both ‘wrong in theory’ and ‘wrong in practice’, and KDS—at least in some settings—feel that the field not only ‘bark[s] up the wrong tree’ but ‘miss[es] the forest entirely’. More specifically, the school of thought, and the concepts and methods within it, are painted as being applicable only to a very narrow range of investigations, to the exclusion of most of the important questions and study designs in modern epidemiology, such as the effects of genetic variants, the study of ethnic and gender disparities and the use of study designs that do not closely mirror randomized controlled trials (RCTs). Furthermore, the concepts and methods are painted as being potentially highly misleading even within this narrow range in which they are deemed applicable. We believe that most of VBP’s and KDS’s criticisms stem from a series of misconceptions about the approach they criticize. In this response, therefore, we aim first to paint a more accurate picture of the formal causal inference approach, and then to outline the key misconceptions underlying VBP’s and KDS’s critiques. KDS in particular criticize directed acyclic graphs (DAGs), using three examples to do so. Their discussion highlights further misconceptions concerning the role of DAGs in causal inference, and so we devote the third section of the paper to addressing these. In our Discussion we present further objections we have to the arguments in the two papers, before concluding that the clarity gained from adopting a rigorous framework is an asset, not an obstacle, to answering more reliably a very wide range of causal questions using data from observational studies of many different designs. VBP characterize the mainstream view within what they call the ‘causal inference movement in epidemiology’ as belonging to the ‘restricted potential outcomes approach’, which they define to be the approach in which only the effects of exposures that correspond to currently humanly feasible interventions can be studied. KDS focus instead on DAGs (rather than potential outcomes) as the main target of their criticism. However, in many places they appear to (wrongly) conflate DAGs and potential ouctomes, and they certainly share the misconception that only currently humanly feasible interventions can be studied within this approach. As we discuss later (see misconception 1), we strongly disagree with this characterization. We also don’t much like the term ‘movement’, and so—for want of a better label, and to avoid cumbersome repetitive descriptions—we’ll call the school of thought that both VBP and KDS have in their sight the ‘Formal Approach to quantitative Causal inference in Epidemiology’, or FACE. In the next sections we describe what we see as the core principles of this approach, with examples of where these have been illuminating and enabled causal analyses under less restrictive assumptions. The broad features that characterize the majority of the work done by the FACE are, having first thought carefully about the nature of the causal question to be addressed, to convert this into a precise quantity to be estimated (i.e. a causal estimand), typically using the notation of potential outcomes. The causal question one ideally wishes to address may often be replaced by a similar causal question that can more feasibly be addressed given the constraints of the data at hand. There is a trade-off here. No one wants ‘the right answer to entirely the wrong question’; indeed, this is what has led the FACE to recommend against ‘retreating into the associational haven’ but rather ‘to take the causal bull by the horns’.3 But presumably equally uncontroversial is the observation that ‘an entirely wrong answer to the right question’ is also futile. Arriving at a good compromise between these two competing concerns is one of the many important tasks facing applied researchers. Explicitly formulating the causal estimand may seem like an obvious first step, but one that is often ignored in applied practice where researchers may jump to modelling associations and presenting their results in terms of, for example, odds ratios or hazard ratios, while foregoing the more interesting and concrete scientific questions such as ‘what would the risk of this outcome be if one could eliminate the exposure?’ This clarity moreover allows one to be rigorous about the assumptions (e.g. consistency, conditional exchangeability and positivity) under which the estimand can be identified from the data at hand, and then for flexible estimation strategies to be developed that are valid under these assumptions. Finally, tools are recommended to assess quantitatively the sensitivity of the results to plausible departures from the assumptions, to aid interpretation, and to discuss possible misinterpretation, of the results. In the Supplementary material (available at IJE online) we give examples of causal estimands and describe the most commonly invoked assumptions for their identification in the context of a simplified investigation of the effect of maternal urinary tract infections during pregnancy on low birthweight. In many settings (problems involving time-dependent confounding and mediation are good examples4–9), the increased formality characteristic of the FACE has highlighted the implausibility of the assumptions (e.g. no ‘feedback’ between exposure and confounder) required for standard analysis strategies to give meaningful answers to the causal questions being posed, and has led to improved alternatives (e.g. g-methods) that are increasingly widely used in practice.10–13 The FACE has moreover given rise to an array of methods for nonlinear instrumental variable analysis14–16 and for nonlinear mediation analysis9,17–23 where only ad hoc and biased approaches existed before. Other examples where this approach has led to new insights and/or methods include the low birthweight and obesity ‘paradoxes’24–27 (see further discussion in ‘Example 2: Birthweight paradox’, below), the comparison of dynamic regimes,28 the impact of measurement error,29,30 noncompliance in clinical trials,31 distinguishing confounding from non-collapsibility32 and many more. More recently, and looking to the future, the advent of omics technologies, electronic health records and other settings that lead to high-dimensional data, means that machine learning approaches to data analysis will become increasingly important in epidemiology. For this to be a successful approach to drawing causal inferences from data, the predictive modelling aspects (to be performed by the machine) must be separated from the subject matter considerations, such as the specification of the estimand of interest, and the encoding of plausible assumptions concerning the structure of the data-generating process (to be performed by humans). Whereas traditional epidemiological approaches to the analysis of data naturally blur the two aspects, the FACE makes the distinction explicit, and hence allows machine learning methods to be successfully employed.33 Its emphasis on definitions and assumptions has sometimes given the false impression that the FACE is a ‘paralysing’ approach. How should the applied epidemiologist proceed in settings where clear definitions are hard and assumptions are violated, but nevertheless quantitative causal inference is needed? The advice that accompanies the theory is pragmatic, for The more precise we the the risk of in some of the study In we a to the that give rise to our within a field to in the range of questions that can be addressed, in the assumptions and in to assess the sensitivity of to in the We two is a of 2: The effect of exposure on as a risk exposure is and this in risk is is of to be We believe that some of the between the and epidemiological on stem from a to the between the two and the different of and required such from is in many settings that effect estimation assumptions on of what is required for the causal for methods that use instrumental There are three main misconceptions on which VBP and KDS their We discuss in This is to much of VBP’s and KDS’s criticisms of the but we do not believe to be a of the within the The FACE having in interventions that are ideally to and this is what is from the by We do not with their from these do we from the in the field more that these interventions be currently humanly of the of the investigation is to The by VBP on for an to be is not that the can be is a between and is in our is the and not In of the work from the FACE makes that the results from a causal analysis to feasible and/or or as the conditional exchangeability the This interventions which are in the if they applied to the exposure to some for they would not the outcome in to have that exposure from what Furthermore, at an can be to a of the and that is possible to proceed even a that a of interventions is The of this on the context for example, is often in given that if a applied to the exposure to some for this would not the conditional of the outcome given in to have that exposure from the conditional of the outcome given these with exposure For example, in an observational study of the effects of the work by and the of causal effect estimated from a observational study other to a that the of many different about the effects of such a is of for but is for scientific of work from the FACE is on important exposures for which certainly no humanly feasible is and often no could be of for which the observational data are For example, the impact of on in with the aim of the risk that would have been such infections been Their analysis to give on these infections are, even no feasible that could for the the authors view their results as being about the effect of This effect may from the effect of an to could be do more than Other exposures that have been studied in this context are, for example, in and and discuss the and in a recent of the There is in the causal framework that the to correspond to a feasible in to the causal assumptions for the is to that the used to define the to a and in the can be further to include an of the impact that would be if that to be in of the recent work from the FACE has been to the study of in particular using and effects have been by they interventions that are, by their very humanly of the being in other no randomized could even in be that would the estimation of these effects under assumptions to by The view within the FACE is that these of the of the epidemiological questions they aim to are of our the very of the interventions they be In to this we by what we believe the of the study to inference about the effect of a exposure is the we the study we would if our concerns only with no for or We believe that such a study would have the many of no that the effect of the exposure in a of different can be as as effects to different if a of if an of for the exposure more than two if is of at that and effects can be data that effect can be and no other of data, noncompliance or measurement is that that the study would be randomized the that the FACE often of ‘the randomized but this that are to be as better than observational studies for causal observational studies in practice are more to to and often also The which has as one of that is is in some to a and in other to a observational by the context can a be made on which is better for that if both are and In many a would be the FACE having in the as a to that the observational study is and in the most This is even more in studies such as that to the dynamic a key between a observational study and the study is that a focus of the methods from the FACE is the observational study can be in such a that the study with to This not to the view that the FACE to observational studies in such a that the results are to that would have been from a on the The aim is to observational studies in such a that the results are to that would have been from the one of which is that the exposure is two are and an investigation of this led to important insights the by of the of the by studies like randomized could be to this of and the of the randomized controlled for other study designs (i.e. observational are then only valid and to the that they This with to has been the of recent in in to and and and We see this a as of a in a as as is about by the different that and have to that both and and which has two lead have as lead and KDS also to the on to and given to with the of of are more with of or not is to a such as from the data at hand. would be very to that and be in of in However, using in of in some is the view within the FACE we that that of would have a risk of they been is meaningful only if the share a to of what they been and this further In the are they to be from in what their being for that are from the of in they to be up in their or similar the conditional many of the In to further must be for the study up in or do we that this is a in a are in what would have to these they been in what would have to these they been they are in one of three possible and as effect gender and ethnic and then in what can be done to as and can be done to define interventions on or the effect of the of and that is in the effect of and this is what KDS about in their third of these interventions on For the would be on the of rather than on We that the FACE is not that and is not these are to many important epidemiological The that KDS from the that causal inference is they have the observation made by the FACE that is to answer the question of ‘what would if we and that in we are more in one of or as that we should not study and even and at of is the of causal that be a is not before on to that is the effect or rather than the effect of that is of to can be from the applied on of for example, that these are using associational for and as as to have a ethnic in less The is is studied. For example, in the on gender a key The of a of on the causal effect of on health in and and that the on should be the for both This on a study of effect by effect is associational with to gender causal with to The question in this context not interventions on VBP discuss the view of such a are to and characterize one of the objections to this as from The the of been no would have been or we to would be As by and the in are for have have from their or their or and if so in what from that the of these different would be the to this but a of the of exposures in the study as by and is to the in particular in to as is the of the FACE given by VBP on from the concerning obesity and that under the of the like obesity or can no be as The by KDS of the role by DAGs in causal inference is to what is in the key and in this and to what is in to causal We therefore, by the role of DAGs in causal inference, before the key misconception that many of KDS’s We this section by further in their discussion of the DAGs to their three As used in DAGs are of conditional The of an between two in a is used to conditional between the two by these two conditional on the by the in the call these conditional The of conditional is that conditional (i.e. conditional between two given sets other than by the in the can be from the conditional used to the an as DAGs are for causal inference the causal effects of can be in terms of conditional between exposure and DAGs as to which conditional characterize the effect of interest, by the causal that would exposure and outcome Causal are by the data-generating which on the of causal the of between the of effects between and study which is not in the data but may be from can be in the causal The DAGs used in causal inference can be some from or the to see for example, a given of is to for confounding given the assumptions in the causal DAGs have very in this process are to have about the of or the of DAGs to the that this process is We that the DAGs used in causal inference a and for example, the paper by in which data can be in different by different causal to the different possible study questions of interest, and subject matter that these In the of the is possible to address KDS’s criticisms of many that data are not to at the at causal inferences by This is and is DAGs are in causal to the assumptions on a explicit, and to the of a into a that is no for hard about the and and that the we to and we Causal DAGs don’t to such a the causal is the of the hard not a for it, and the is which the next in the from the of this hard to a of their criticisms are similar and from the underlying for they can a into what be We of is the that to the and not KDS that the is to to the of the exposure in question would to an and However, the many examples from the FACE have that even the DAGs are they do much into the in We the discussion by KDS of their three examples rather to the DAGs they to are not This in to the of DAGs for clarity of thought and in these the by KDS in to the effect of on The this is that low birthweight be to a of one of these be maternal be of or a we the low birthweight then is that is the of low birthweight. we the low birthweight do not then we maternal is as a for low so that must have been some other such as or a the of which for are much not for the of low birthweight and we are up an comparison between the and we could for such the associations for the and as possible KDS during their to and much than by that birthweight as a of genetic or the or to or during and this not a more of the of in the of a in which could also a In other the and KDS’s are the and indeed, the only that such a may KDS’s more We don’t their therefore, that the is while the is Their having identified the potential for in a is matter to the do and if they are to the is of entirely This is having identified the that the could be in this the FACE on to or not plausible for the effects of such on birthweight and would to the In DAGs are the from subject matter the the data analysis and/or sensitivity but has the FACE made to this As we under KDS are in with the FACE in their discussion of their third example, interventions on don’t from of the specification that interventions on and in the that they criticize. than that the FACE is up the wrong and the forest KDS should aim this at their of the such as are the the causal effects of and the FACE has the in and entirely that is to be the question of In view of the of causal on observational data, have to only of VBP that the FACE has been a to this to the associational is this has to in a of in data analyses are to or the of a formal framework makes to between analysis strategies that target the causal from that do The has been in analysis strategies that to even in the where confounding are be to from the many possible associations between exposure and outcome that one could the one that the causal at the FACE has the of such effect of can be assumptions can be and analysis strategies developed that are valid these assumptions are The FACE to a framework under which causal can be not the many of epidemiological such as data, but rather to under what such causal to be are examples of this work by the FACE in to data and In to that a good of a effect a of the exposure and effect the framework as a we have that the formality that the FACE not the of humanly feasible as in the by the We believe that many epidemiological that aim to the impact of health have such interventions in of causal have to from the mainstream approach as by not using potential of in particular the have been in some assumptions in approaches on potential or are The framework to the principles even more of the causal target of estimation and the assumptions under which this can be in terms of data the approach results from the potential outcomes approach, and we view as a of the FACE. Other causal in their to avoid potential have to be less explicit, and VBP and KDS recommend that other for be in epidemiology. We that their which are not to be will not into the VBP and KDS the for the of studies and We with and view the concepts and methods of the FACE as rather than this in two more causal analyses of the studies to a the of the and by being clear what question is being addressed, and under what assumptions the analysis used can be deemed from different studies can be more reliably We a recent of where a to on in both these VBP and KDS the analysis of data, the use of and the of but as we have these are done within the the section of the FACE to sensitivity analyses has at core the at least or of approach to the of is by VBP also that framework is more to epidemiological Whereas of we view the framework as belonging to the is that the framework is more in terms of the assumptions makes than within the are assumptions similar to of the only with to interventions on the the assumptions with to interventions on variable in the causal We to VBP be to this more restrictive while the framework that as VBP and KDS to a examples from in which successful causal inferences the formality by the FACE. We should be of on these the a similar would lead one to that not a formal theory at are many in where and formal The in this is that no is given to the many examples where and have been This not that approaches have no they should and do the of studies and but for a formal theory and approach. We view the FACE as formal tools to are by what KDS call to the is often one to the in the first associations can be in to or not and for in mediation analysis and instrumental variable as as nonlinear are at is no question in our that a formal theory is to data aspects of the FACE have been by Its to be about assumptions has often been as if this framework more assumptions than traditional This has then led to use the from which they where causal is only under even assumptions. by VBP and KDS further misconceptions if would that many important exposures would be from being studied within the FACE framework and many such as causal as In this response, we have to these misconceptions while the clarity that from having a rigorous framework on clear definitions and assumptions, we have highlighted the that should and do the theory applied in with the role by subject matter We are to about these and to be to that the FACE not epidemiological questions that be to humanly feasible epidemiological designs that aspects of randomized and that or methods the of subject matter the FACE to on what can be about these questions and from these designs under the most plausible assumptions given the data, and subject matter at hand. As in a recent on similar to or not data can be used in causal to this of the of and the strategies may be the only data the only data that we can are we believe that data will the for we must the use of data rather than into using the data Supplementary data are at IJE is by a by the and the The for is by the from from the We are to Vandenbroucke, and for on these and/or on an of