Expertise in evaluating second language compositions

This study (1) assesses whether raters implicitly distinguish students' writing expertise and second language proficiency while evaluating ESL compositions holistically and (2) seeks to describe the decision-making behaviours used by experienced and inexperienced raters in this process. The performance of 7 novice and 6 expert ESL teachers was assessed while they evaluated 12 compositions written by adult students with differing levels of ESL proficiency (intermediate and advanced) and writing expertise (average and professionally experienced writers) in their mother tongues. Multivariate analyses of rating scores indicated that both groups' evaluations distinguished students' second language proficiency and writing skills as separate, non-interacting factors. Descriptive analyses of the raters' concurrent verbal reports revealed 28 common decision-making behaviours, many of which varied significantly in use between novice and expert groups. Implications discuss biases in holistic evaluations of second language compositions, aspects of expertise in this skill, and potential uses of this research for the training of composition raters and student-teachers.