Answers are grounded in this paper and cite their sources.
This study describes a simple set of statistical parameters for assessing the reliability and validity of oral examinations (OE). Traditional feedback to examiners tends to categorize them as 'hawks' or 'doves' on the basis of whether their personal mean mark is above or below the group mean. Our study shows that the mean OE mark on its own is not a good measure of examiner performance. We suggest that inter-rater reliability between examiner pairs is a more satisfactory indicator of reliability and face validity. The correlation between the OE marks given by an examiner and the student's subtotal from written parts of the exam (SUBTOT) is suggested as a useful indicator of OE validity. These measures, as applied to our own student exam results, suggest that our OE examiners are performing at an acceptable standard of agreement (Cohen's Kappa for pass/fail 0.74, p < 0.0001), and support the use of the OE as a method of student assessment.
No takes yet. Share an insight, caveat, or question.
Allison et al. (1992) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: