An Experiment on Improving Response Rates and Its Unintended Impact on Survey Error

Groves (2007) warns that the "[b]lind pursuit of high response rates in probability samples is unwise" (p. 668) because it may have the unintended consequence of actually increasing survey error. This will happen when efforts to improve the response rate increase the correlation between the propensity to respond and the survey variable being measured. The findings of a study we conducted a while back, in 1997, provide a good illustration of Groves' point. This experiment was designed to test the impact of factors hypothesized to increase the response rate but had the unanticipated effect of increasing survey error.