Methods for pre-testing and piloting survey questions: illustrations from the KENQOL survey of health-related quality of life

Part of the assertion that any survey researcher can make about the validity of their results needs to contain an analysis of questions and their responses from the respondent's viewpoint. Claims concerning the validity, reliability and sensitivity of health-related quality of life measures tend to be based on the quantitative approach of psychometrics, which fails to identify when respondents: misinterpret questions; do not recall the information requested; or give answers that present themselves in a better or worse light. The paper presents some approaches to pre-testing and piloting survey questionnaires to check the interpretation of survey questions, using illustrations from the KENQOL project. The paper describes: how the intended referential and connotative meaning of each question was established; the criteria to judge the appropriateness of each question; the methods used to make those judgements; and the process of reviewing questions based on findings. The role of piloting is highlighted, and further reading is suggested for readers wishing to develop a model for their own investigation.