Key points are not available for this paper at this time.
Abstract A framework is proposed for the analysis of ordinal categorical data when ratings from several judges are available. I emphasize the tasks of estimating latent trait characteristics of individual items, regressing these latent traits on observed covariates, and comparing the performance of raters. The model is illustrated in the design and evaluation of an automated essay grader. This grader is based on a regression of variables, obtained from a grammar checker, on essay scores estimated from a panel of experts. The performance of the grader is evaluated relative to human graders, and implications on the reliability and repeatability of both automated and human raters is investigated.
Valen E. Johnson (Fri,) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: