A Novel Technique for Comparing the Reliability of Multiple Peer Assessments with that of Single Teacher Assessments of Group Process Work

There have been no studies which compare the reliabilities of peer and teacher ratings of group process work. This is mainly because there is usually just one teacher who is in the position to rate the work of a group over time. Hence it is not possible in these circumstances to determine the reliability of teacher ratings. The reliability of multiple peer ratings can be estimated, however, through traditional analysis of variance (ANOVA) methods. This paper employs a novel application of ANOVA techniques to compare the reliability of multiple peer ratings with single tutor ratings. The method is outlined and demonstrated by application to rating data from two different courses. Both involved sets of multiple peer and individual tutor ratings from parallel classes that were used to assess student contributions to group process work. Implications for practitioners and researchers are discussed in terms both of the application of the technique and the findings from the two case studies.