A plea for new psychometric models in educational assessment

OBJECTIVE: To describe the weaknesses of the current psychometric approach to assessment as a scientific model. DISCUSSION: The current psychometric model has played a major role in improving the quality of assessment of medical competence. It is becoming increasingly difficult, however, to apply this model to modern assessment methods. The central assumption in the current model is that medical competence can be subdivided into separate measurable stable and generic traits. This assumption has several far-reaching implications. Perhaps the most important is that it requires a numerical and reductionist approach, and that aspects such as fairness, defensibility and credibility are by necessity mainly translated into reliability and construct validity. These approaches are more and more difficult to align with modern assessment approaches such as mini-CEX, 360-degree feedback and portfolios. This paper describes some of the weaknesses of the psychometric model and aims to open a discussion on a conceptually different statistical approach to quality of assessment. FUTURE DIRECTIONS: We hope that the discussion opened by this paper will lead to the development of a conceptually different statistical approach to quality of assessment. A probabilistic or Bayesian approach would be worth exploring.