Video Listening Tests: What Are They Measuring?

Abstract This article examines how eight advanced ESL learners attend to and utilize the nonverbal information found in a video text. Using verbal report methodology in which test-takers verbalized aloud their thought processes while taking a video listening test, the results suggested that the test-takers varied in how they attended to and utilized the nonverbal information provided by the video texts. In addition, the data provided evidence that individual test-takers varied in their ability to utilize the nonverbal information both to make meaning of the spoken text and to answer the comprehension items associated with the text. The implications for a construct definition of listening ability that includes the ability to utilize the nonverbal information found in most spoken texts is discussed. Acknowledgments The author would like to thank Jim Purpura, Leslie Beebe, Barbara Hawkins, Jane Monroe, Jee Wha Dakin, and Santoi Wagner for the contributions to this study, and the three anonymous reviewers at LAQ for their valuable feedback. Notes 1 CitationCohen (1996) and CitationEricsson and Simon (1993) suggested that these types of questions might be necessary to obtain more complete and informative data. Although there is the danger that these more probing questions could change the thought processes of the participant, these types of questions can also "offer a rich enough source of information not available through think aloud protocols alone that researchers are willing to risk threats to the reliability of the verbal report tasks in order to obtain the data" (CitationCohen, 1996, p. 11). 2At this point in the verbal report procedure, it seemed that Hyunmi had become fixated on coming up with the word wrist. Although the researcher tried to be as unobtrusive as possible to minimize any reactive effect, it was decided to provide the word for her so that she could continue with the reporting. 3However, the think aloud procedure used here was similar to CitationLeow and Morgan-Short's (2003) procedure, in which the researchers found that having the participants think aloud while taking a reading comprehension test did not have a statistically significant effect on their performance on the test. 1M. Wagner is the author under his legal name.