Answers are grounded in this paper and cite their sources.
Student evaluations of teaching (SETs) remain widely used in consequential institutional decisions, yet their interpretation typically relies on mean scores that obscure differences in student agreement and the contexts in which evaluations are produced. This study examines evaluation level and evaluative heterogeneity as distinct but complementary properties of SET evidence and investigates how each is structured across instructors, teaching contexts, and time. We analyze four years of university-wide administrative SET data using cross-classified and within-between models, complemented by context-adjusted institutional classifications. Evaluation level showed substantially greater instructor-associated clustering than heterogeneity, whereas heterogeneity was predominantly occasion-specific. Teaching conditions were differently associated with the two properties, and demographic associations were context dependent rather than uniform. Contextual adjustment rarely changed institutional classifications but mattered primarily near the decision boundary, where similar mean scores could rest on different evidential profiles. These findings show that a SET mean does not fully characterize the evidence supporting it. Response volume, evaluative heterogeneity, teaching context, and longitudinal stability can qualify the evidential weight attached to mean ratings, particularly when SETs inform consequential institutional decisions.
No takes yet. Share an insight, caveat, or question.
Castro et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: