Comparison among methods for improving response quality of surveys

Certain participants are insincere when responding to questionnaires. Two current approaches for detecting unmotivated or dishonest respondents, the instructional manipulation check (IMC), and the seriousness check, were examined. We also attempted to improve the quality of survey responses by asking respondents to take an oath that they would be serious before they started answering the questionnaire (TO). The respondents in two Web surveys were randomly assigned to one of four versions of the questionnaire. The main results indicated that (a) response quality tended to improve when respondents who did not follow instructions were excluded from the sample; and (b) respondents that who took an oath to answer seriously chose fewer “don’t know” options, straight line responses, and midpoint responses than the control group, suggesting that respondents behaved consistently with their initial commitment. The results indicate that although IMC is superior for improving data quality, techniques for deterring less serious responses including TO were desirable in that they did not reduce the sample size.