The Effects of Non-Response on Statistical Inference

Surveys have been, and will most likely continue to be, the source of data for many empirical articles. Likewise, the difficulty of making valid statistical inferences in the face of missing data will continue to plague researchers. In an ideal situation, all potential survey participants would respond; in reality, the goal of an 80 to 90% response rate is very difficult to achieve. When nonresponse is systematic, the combination of low response rate and systematic differences can severely bias inferences that are made by the researcher to the population. It is important for the researcher to assess the potential causes of nonresponse and the differences between the observed values in the sample compared to what may have been gained if the sample was complete, particularly when the response rate is low. There are methods available that substitute imputed values for missing data, but these methods are useless if the researcher lacks knowledge of how the responders and nonresponders may differ. With regard to statistical inference, the researcher also should be aware of the difference between a convenient sample and a probability sample. Valid statistical inference assumes that the probability of characteristics observed in the sample bear some relationship to their occurrence in the population. For example, in a simple random sample each member of the accessible population has an equal chance of inclusion in the sample. A convenient sample lacks the statistical properties of a probability sample that allow the validity of its inferences to be assessed strictly from a mathematical framework. The context of the research and the type of data being gathered greatly affect the validity of any generalizations the researcher makes with regard to the population the convenient sample attempts to represent.