How Definitive Are Conclusions Based on Survey Data: Estimating Robustness to Nonresponse

The problem of nonresponse poses a threat to the external validity of survey conclusions. In this article, procedures are developed to estimate the robustness of survey conclusions to nonresponse by drawing an analogy between studies unavailable for meta‐analysis and nonrespondents in a survey. We extended the procedure to assess the effects of systematic differences between respondents and nonrespondents on t‐ and F ‐values used to test group differences. Steps to construct robustness tables that are independent of sample size, survey length, and scale width are outlined to determine how definitive conclusions are from survey data.