Psychology examiners re-examined: A 5-year perspective

Reliability and consistency of examination marking were examined across 5 years of a large second year psychology course (approximately 4800 examination answers). The analysis uses single-marked answers to assess variation in marker reliability and identify factors that influence this. Overall reliability of the examination ranged from 0.75 to 0.87. Reliability of marking was similar to that reported with double-marking and marks for specific questions were consistent with marks averaged over the remainder of the examination (r = 0.5 approx.) However, there was variability in consistency across questions and markers. Most variability reflected sample size, but there was also variation across years and between areas of psychology in the reliability and consistency of mark allocation. The origins of this variability need to be understood, while the existence of variation raises the possibility that overall reliability can be increased.