Can algorithms Replace Expert Knowledge for Causal Inference? A Case Study on Novice Use of Causal Discovery

Abstract Background With growing interest in causal inference and machine learning among epidemiologists, there is increasing discussion of causal discovery algorithms for guiding covariate selection. We present a case study of novice application of causal discovery tools and attempt to validate the results against a well-established causal relationship. Methods As a case study, we attempted causal discovery of relationships relevant to the effect of adherence on mortality in the placebo arm of the Coronary Drug Project (CDP) dataset. We used four algorithms available as existing software implementations and varied several model inputs. Results We identified 15 adjustment sets from 17 model parameterizations. When applied to a baseline covariate adjustment analysis, these 15 adjustment sets returned effect estimates with similar magnitude and direction of bias as prior published results. When using methods to control for time-varying confounding, there was generally more residual bias than compared to expert-selected adjustment sets. Conclusion Although causal discovery algorithms can perform on par with expert knowledge, we do not recommend novice use of causal discovery without the input of experts in causal discovery. Expert support is recommended to aid in choosing the algorithm, selecting input parameters, assessing underlying assumptions, and finalizing selection of the adjustment variables.