What question did this study set out to answer?

The paper evaluates the impact of survey mode on responses to sensitive questions, comparing Push-to-Web surveys and face-to-face interviews.

April 12, 20260 citationsOpen Access

Responding as Expected? The Effects of Survey Mode on Estimates of Sensitive Attitudes in Self-Completion and Face-To-Face Interviews of the European Social Survey

BSBlanka Szeitl BSBence Ságvári VMVera Messing

Key Points

The paper evaluates the impact of survey mode on responses to sensitive questions, comparing Push-to-Web surveys and face-to-face interviews.
Analyzed responses from two Push-to-Web surveys and corresponding face-to-face ESS surveys in Hungary.
Focused on culturally sensitive topics: attitudes towards gay and lesbian individuals and immigrants.
Utilized one-, two-, and multidimensional analyses including GLM models to assess response patterns.
Push-to-Web surveys showed similar response rates to face-to-face surveys but attracted more educated and older respondents.
Post-stratification did not fully correct for differences in attitudes towards gay and lesbian individuals and immigrants between survey modes.
Survey mode significantly influenced how respondents reported their attitudes, overshadowing demographic factors.

Abstract

Conducting face-to-face surveys is becoming increasingly challenging. The shifting technological and social environment of survey data collection is leading researchers to seek more sustainable solutions through new data collection techniques and rethinking of traditional survey methods. Recent examples are the Push-to-Web (PtW) survey experiments initiated by the European Social Survey (ESS) research consortium, which using probability sampling reach respondents through postal invitation and combines web-, and paper-based self-completion questionnaires. The literature indicates that the method of data collection may have a strong impact on who participates in a survey and how people answer the questions. These two phenomena are combined in the concept of the mode effect. This paper evaluates the PtW surveys in Hungary from the perspective of mode effects by pairing two PtW surveys with face-to-face ESS surveys closest in time. The evaluation is made along the sample compositions and the answers to two culturally and socially sensitive questions for which the different survey situation is seen as a highly relevant factor due to the specific political context in Hungary: attitudes towards (1) gay and lesbian, and (2) immigrants. Answers to these questions from both surveys are assessed in one-, two-, and multidimensional (GLM models) analyses. The results show consistent patterns in the analyses of both survey pairs. (1) PtW surveys yield similar response rates as the face-to-face surveys but tend to appeal to the more highly educated and older sections of the population. (2) Despite the correction of sample composition by post-stratification, the results reflect different attitudes towards gay and lesbian, and immigrants in the two survey modes. (3) The survey mode has an independent impact on sensitive measurements: in GLM models, the survey mode has a significant effect on how respondents report their attitudes, and overwrites the original demographic correlations.

问 AI

Bookmark

View Full Paper