Key result
Spanish university entrance exam boards demonstrate large rating inconsistencies with ~1 logit difference between extremes.
Why the study?
The study was conducted to address the need to measure the progress of rater performance in University Entrance Examinations.
Observational (n=3,000)
Yes
The study highlights significant inconsistencies in the rating process of University Entrance Examinations in Spain, indicating a need for better criteria to ensure comparability.
Answers are grounded in this paper and cite their sources.
May compromise equity in admissions decisions; leaves open whether standardized criteria reduce board variability.
Introduction: University Entrance Examinations (EBAU in Spanish) is a nationwide program for assessing student achievement levels in Spain and determining access to public undergraduate degrees. Considering the need to measure the progress of rater performance, this paper analyses rating data from the June 2018 exam sitting in the Valencian Community, Spain. Methodology: A total of 54 assessment boards and 3000 students from five public universities were observed. The Many-Facet Rasch model (MFRM) was used as an extension of the one-parameter Rasch measurement model. All facets involved in analyses (examination board severity, subject difficulty, and group performance) were located on a common underlying linear scale. Results: Results showed large inconsistencies in the rating process with differences in severity levels of many subjects both within and between universities. Moreover, extreme differences were detected among subjects associated with health or biological sciences (Chemistry, Physics) as well as those related to the Humanities and Social Sciences (Latin or History of Spain). Conclusions: This study may serve as a starting point toward a reflective debate on the need to apply better criteria ensuring comparability of examination standards in Spain.
No takes yet. Share an insight, caveat, or question.
Veas et al. (2024) conducted an observational in University Entrance Examinations (EBAU) (n=3,000). Assessment boards was evaluated on Severity and leniency levels of examination boards measured in logits. Assessment boards in the Spanish University Entrance Examinations demonstrated large inconsistencies in rating severity, with differences of almost 1 logit between the most and least severe boards.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: