An Evaluation of Alternative Item Formats for Testing English as a Foreign Language

Responses to general background questions, by percentages Page 40 2. Self-reported grades in English as a foreign language 3. Self-ratings on English language skills .• .• • • • • 4. Means and standard deviations of multiple-choice measures • 5. Reliability indices of multiple-choice measures • .• • • • 6. Means, standard deviations, and reliabilities of open-ended objective measures and of subjective measures 41 42 43 46 48 7. 8. Observed intercorrelations among multiple-choice measures • Intercorrelations among multiple-choice measures, corrected for attenuation • • • . . .• • • • • .• • • • • 0 • 54 56 9. Observed and corrected intercorrelations among regular TOEFL subsections .• • • . .• • • • • • • • • • • • • .• • 60 10.Observed intercorrelations among open-ended objective measures and subjective measures . .• . . .• • • . .• .• .• . .• 62 11.Intercorrelations among open-ended objective measures and subjective measures, corrected for attenuation.• • • • . .• 0 64 12. 13.Observed correlations between multiple-choice measures, and open-ended objective measures and subjective measures Correlations between multiple-choice measures, and open-ended objective measures and subjective measures, corrected for attenuation 0 • • • • • • • • • • • • • • • • • • • vi 66 67 PURPOSE AND BACKGROUND A major purpose of the Test of English as a Foreign Language (TOEFL) is to provide information useful to colleges and universities in making decisions regarding the admission, placement, and possible assignment to special language instruction of foreign students planning to study in the United States and Canada.Its unique role is to assess the foreign student's competence in English that he will need in order to successfully pursue a program of studies at a college or university where English is the medium of instruction.The overall purpose of the present study was to obtain information useful for evaluating and revising TOEFL content and content specifications.To achieve this purpose, questions of validation, criterion selection, and content specifications were investigated.The tasks involved in carrying out this study were developmental as well as evaluative.Thus, four objective subtests that might increase the effectiveness of TOEFL, and tailored criterion measures of English productive skills, speaking and writing, were developed.Also included in the study were two open-ended response measures which have shown particular promise for testing English as a foreign language.One is a rewriting task used by Kellogg Hunt (197Ga, 197Gb), and the other is the "Cloze procedure" task (Taylor, 1953), in which subjects are instructed to replace words that have been deleted from prose passages.Validation, criterion selection, content specifications, and related questions regarding TOEFL will be considered first.The subtests used in carrying out the study and the data resulting from the study will be presented in later sections of this paper.