Selecting Confounders from Covariates

SUMMARY Most analyses of observational studies to investigate the relationship between a risk factor and an outcome use some data-dependent method to select a subset of the available covariates to include as confounders. This paper evaluates methods which can be used to make this selection in linear models. The methods are applied to the selection of covariates in a study of the effect of lead in the blood on children's ability and are evaluated by a simulation study on multivariate normal data. None of the methods can be guaranteed to give satisfactory results in all cases, and all the methods can result in misleading inferences. Recommendations for the design and analysis of observational studies are made in the light of these results. Selection procedures for variables, such as stepwise regression methods, are commonly used in the analysis of observational studies. They are the default option in the regression procedures of most statistical packages and, in my experience, most users of such packages believe that good statistical practice requires their use. Their most obvious role is as part of exploratory data analysis, to seek out the most important independent variables. But it is also common to use a selection procedure before estimating a parameter or making a prediction, and then to work from the chosen model as if it had been chosen a priori. The general problem of statistical inferences after preliminary tests has long been recognized (e.g. Bancroft (1964)) and guidelines for some problems were investigated in the 1960s, when it was expected that the availability of high speed computers would help to determine when preliminary tests (like those used to select variables) would give increased power without compromising the validity of the final inference. Reality has turned out somewhat differently. High speed computers have increased the number of preliminary tests that are carried out, largely ignoring their influence on the true size of the final test or confidence interval. The selection of a subset of variables in a linear model for the prediction of future values of the dependent variable has been the focus of many statistical papers. Much of this work, reviewed by Hocking (1976) and Thompson (1978a, b), has been concerned with deriving mean-square error of prediction (MSEP) criteria that can be calculated from the data and have expectations equal to the MSEP. Several statistical packages now include a regression option that uses a leaps and bounds