Using Level-of-Effort Paradata in Non-Response Adjustments with Application to Field Surveys

Summary The paper considers the use of level-of-effort (LOE) paradata to model the non-response mechanism in surveys and to adjust for non-response bias, particularly bias that is missing not at random or non-ignorable. Our approach is based on an unconditional maximum likelihood estimation (call-back) model that adapts and extends the prior work to handle the complexities that are encountered for large-scale field surveys. A test of the ‘missingness at random’ assumption is also proposed that can be applied to essentially any survey when LOE data are available. The non-response adjustment and the test for missingness at random are then applied and evaluated for a large-scale field survey—the US National Survey on Drug Use and Health. Although evidence on non-ignorable non-response bias was found for this survey, the call-back model could not remove it. One likely explanation of this result is error in the LOE data. This possibility is explored and supported by a field investigation and simulation study informed by data obtained on LOE errors.