Quantifying Potential Selection Bias in Observational Research: Simulations and Analyses Exploring Religion and Depression Using a Prospective UK Cohort Study (ALSPAC)

A major problem in research is non-random participation, which can occur either at recruitment into, or loss to follow-up during, the study. In both cases the analytic sample differs from the target population, potentially resulting in selection bias and erroneous inferences. The current study (ALSPAC; Avon Longitudinal Study of Parents and Children) was broadly representative of the target population during recruitment. However, continued participation after 30 years is known to be non-random, including by factors such as religiosity and mental health. We therefore conducted a simulation study comparing 36 different selection mechanisms for missing data, using realistic effect sizes, between an exposure of regular religious attendance and the outcome of depression to estimate the impact of selection bias. Despite non-random participation by the exposure and outcome, given these plausible selection scenarios there was little bias across all models with broadly nominal (i.e., 95%) coverage of the confidence intervals. Thus, given the strengths and patterns of selection explored in this study, the resulting bias could be negligible. However, this cannot be assumed to hold for all studies and must be explored on a case-by-case basis. These methods can be applied as sensitivity analysis to explore the impact of potential selection bias.