What are the characteristics of natural cloze tests?

This study investigates the characteristics of natural cloze tests. Natural cloze tests are defined here as cloze procedures developed without intercession based on the test developer's knowledge and intuitions about passage difficulty, suitable topics, etc. (i.e., the criteria which are often used to select a cloze passage appropriate for a particular group of students). Fifty reading passages were randomly selected from an American public library. Each passage was made into a 30-item cloze test (every twelfth word deletion). The subjects were 2298 EFL students from 18 colleges and universities in Japan. Each student completed one of the 30-item cloze tests. The 50 cloze tests were randomly administered across all of the subjects so that any variations in statistical characteristics could be assumed to be due to other than sampling differences. The students also took a 10-item cloze test that was common to all students. The 50 cloze tests were compared in terms of descriptive, reliability and validity testing characteristics. The results indicate that natural cloze tests are not necessarily well-centred, reliable and valid. A typical natural cloze is described, but considerable variations were also found in the characteristics of these cloze tests (with many of them having skewed distributions and/or poor reliability). The implications for cloze test construction and use are discussed.