Why Choice of Survey Mode Makes a Difference

Don A. Dillman, MS, PhD* Recently, a surveyor who directs a large monthly survey of a general population called to tell me that a simple question he asked on marital status was produc ing different responses over the Internet than when asked by telephone. In particular, he reported that the percent of respondents who answered had consistently declined by about 10% on the Internet version. The reason for his call was to ask me why the web survey was being answered by more unmar ried people than the telephone survey. I responded by asking to see the exact wording of the question for both modes. Over the telephone the respondent had been read this open-ended ques tion: What is your marital status? The interviewer then coded the respondent's answer into one of the listed categories: single, married, separated, divorced, or widowed. Although identical wording was used for the query on the web, the respondents needed to answer by reading and marking one of the five answer categories. Further examination of the data showed that the percent of married respondents had also decreased slightly, while the percent separated had in creased significantly and the divorced and widowed categories had each in creased a little. The likely reason for these differences seemed apparent. When asked one's marital status in an open-ended fashion, single or married are the usual responses one would provide to a casual inquiry from a stranger. The remaining three categories?separated, divorced, or widowed?are more spe cific responses, which if available, some respondents would choose, but would otherwise not feel a need to volunteer. There are many reasons in this age of increased survey alternatives that differences are frequently observed in responses across different modes. Yet one of the most frequent is that surveyors inadvertently or sometimes inten tionally change how survey questions get asked and answered to better fit the particular survey mode. When I pointed out the difference between offering both the query and response choices on the web, but only the former by telephone, he responded by saying that was the standard way of asking this question over the telephone: Our interviewers prefer it that way. The asking of many kinds of questions is affected by mode choice. In earlier years when face-to-face to interviews were our predominant data collection