On the reliability of high-stakes teacher assessment

For a number of reasons, increasing reliance is being placed on teacher assessment in high-stakes contexts in many countries around the world. Simultaneously, countries that have for some time relied to greater or lesser degrees on teacher assessment for high-stakes purposes are in the process of questioning the validity of that reliance. In principle, teacher assessment has an important role to play in increasing assessment validity by complementing testing to cover subject domains more comprehensively than otherwise would be possible. But what is the evidence regarding the reliability of teacher assessment in high-stakes contexts? The answer is that the evidence is limited and often ambiguous. Research has revealed that teachers can be influenced by a number of construct-irrelevant factors as they work towards their judgements, factors such as gender, socio-economic background, effort and behaviour, that risk biasing their assessments. And when considering construct-relevant achievement evidence teachers are often expected to use verbal or semi-verbal sets of criteria, such as level descriptions, which typically require a degree of subjective interpretation in application and so are themselves a source of unwanted variation in judging standards. Arguably the most effective strategy for addressing these issues is participation in consensus moderation. Yet there have been few attempts to provide evidence of the effectiveness of moderation in practice. The potential value of, and the growing reliance upon, teacher assessment in high-stakes applications demand that evaluation of consensus moderation become a built-in part of the process.