Does the testing method make a difference? The case of reading comprehension

The paper reports results of a study which examined the effect that various methods used for testing reading comprehension have on the measurement of that trait. The methods investigated in this research were multiple-choice and open-ended questions, each presented in L1 and L2, on the same L2 texts. Results indicated that each of the testing methods produced different degrees of difficulty for the test takers, and that each of the variables - method, text and language - had a significant effect on students' scores in reading comprehension. These effects were strongest on low-level students. Recommendations are made with regard to the use of multiple procedures for testing reading comprehension as well as to the need for further research towards finding the most valid method for testing that trait.