The reliability of marking on a psychology degree

The reliability of marking for the final cohort of students to graduate from the psychology degree scheme in place at the University of Northumbria at Newcastle between 1985 and 1993 was investigated. Inter‐marker correlations for some course components were low, but the correlation between students' overall first marks and their overall second marks was .93, a value in keeping with those typically reported for national school examinations. The reliability of a student's overall agreed mark was estimated to be .96 and the standard error of measurement to be about 1 per cent. Further analyses went on to consider the influence of question and option choice on reliability, the representativeness of the cohort studied and the effects of agreeing marks rather than simply averaging first and second marks. Cronbach's alpha was proposed as a means of estimating reliability in the absence of second marking and was used to compare the reliability of first and second markers. The possibility of second marking the work only of those students who were classified as borderline on the basis of their first marks was discussed. The paper concludes with a reminder that reliability does not guarantee validity.