Assessing the content validity of questionnaire items on cognitive activation in English language teaching at primary level through cognitive interviewing

Student and teacher feedback on instructional quality is considered a tried and tested source of data. However, Lenske (2016) demonstrates that students at primary level often fail at understanding and interpreting questionnaire items on instructional quality – student activation, in particular – as intended. These findings emphasize the need for explicitly addressing issues of content validity in student questionnaires. Studies on the content validity of teacher ratings of instructional quality have not been conducted yet. This paper presents a qualitative study investigating the content validity of newly constructed items on cognitive activation in English language teaching at primary level. Cognitive interviews were conducted with nine primary English language teachers and 39 fourth year students to explore how they process questionnaire items on cognitive activation. The interview data was analyzed using qualitative content analysis. Thus, the study replicates Lenske’s (2016) approach and adapts it to teachers as a new target group. The findings show that both groups of participants misinterpreted more than half of the items each. Building on these findings, the paper suggests strategies for ensuring the content validity of items.