Answers are grounded in this paper and cite their sources.
Explores group anchoring's effects on assessment accuracy in high-school musical performances, suggesting potential benefits and limitations.
Sparse rating designs, where it is not possible for all raters to score all students, are often unavoidable in rater‐mediated assessment settings. Unfortunately, sparse ratings can lead to imprecise and inaccurate estimates of model parameters, including student ability, and reduce the detection of rater effects. In the context of latent trait models, researchers have proposed group anchoring as a potential post‐data‐collection solution to sparseness. We used a simulation study and real‐data analysis of ratings from 13 raters of 75 high‐school student musical performances to explore the impact of group anchoring on parameter estimates when compared with various disconnected and sparse rating designs. We manipulated the conditions of student sample size, rater effect presence, and rating design. Our results showed that student parameter correlations between fully connected estimates and group anchoring estimates were generally comparable to those from partially connected rating designs, especially with larger sample sizes. However, the results were less stable for estimating rater parameters. Although group anchoring may serve as a robust alternative to fully connected designs for student achievement, it may not always be appropriate when evaluating rater severity.
No takes yet. Share an insight, caveat, or question.
Wind et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: