Randomized trial explores group anchoring's effects on assessment accuracy in high-school musical performances, suggesting potential benefits and limitations.
Key Points
The study aims to evaluate the effectiveness of group anchoring in improving parameter estimates in rater-mediated assessments with sparse designs.
Conducted a simulation study and analyzed real data from 13 raters scoring 75 high-school students' musical performances.
Manipulated conditions regarding student sample size, presence of rater effects, and rating designs.
Compared parameter estimates from group anchoring with those from disconnected and partially connected rating designs.
Student parameter correlations between fully connected estimates and group anchoring estimates were comparable, especially with larger sample sizes.
Results for estimating rater parameters were less stable compared to student parameters.
Group anchoring serves as a robust alternative for assessing student achievement, but may not be suitable for evaluating rater severity.