Effects of training on raters of ESL compositions

Several effects of training on composition raters have been hypothesized but not investigated empirically. This article presents an analysis of the verbal protocols of four inexperienced raters of ESL placement compositions scoring the same essays both before and after rater training. The verbal protocols show that training clarified the intended scoring criteria for raters, modified their expectations of student writing and provided a reference group of other raters with which raters could compare themselves, although agreement with peers was not an over-riding concern. These results are generally in accordance with hypothesized effects of rater training.