A Comparison of Two Methods of Reading Essay Examinations

The study reported here was designed to explore in detail the relationships within and between readings of Advanced Placement American history papers (CEEB, 1966) when both analytical and holistic methods of reading were used. Ideally, it would have been desirable to have a large number of readers and a large number of papers. The field situation did not permit this. Readers had been engaged for a limited period of time, and the operational reading had to be completed. It was possible, however, to conduct a small study using the four readers assigned to the reading of one of the questions in Form LBP of the examination. The investigators were interested in exploring as many aspects of the relationships as possible, and as the planning developed it became clear that with only four readers there would be difficulties in achieving a single design which would provide the information sought. They finally decided to run two experiments in an interleaved design as set forth in Table 1. One hundred twenty papers were selected for the experiments and divided into eight groups of 15 papers each, designated by the letters A-H in Table i. Groups A-D were assigned to Experiment