Exploring Relationships Between Qualitative Student Evaluation Comments and Quantitative Instructor Ratings: A Structural Topic Modeling Framework

This study demonstrates how Structural Topic Modeling (STM) can be used to analyze qualitative student comments in conjunction with quantitative student evaluation of teaching (SET) scores, providing a scalable framework for interpreting student evaluations of teaching. Drawing on 286,203 open-ended comments collected over fourteen years at a large U.S. research university, we identify eleven latent topics that characterize how students describe instructional experiences. Unlike traditional topic modeling methods, STM allows us to examine how topic prevalence varies with course and instructor attributes, including instructor gender, course discipline, enrollment size, and numeric SET scores. To illustrate the utility of the model, we show that topic prevalence aligns with SET ratings in expected ways and that students associate specific teaching attributes with instructor gender, though the effects are relatively small. Importantly, the direction and strength of topic–SET correlations are consistent across male and female instructors, suggesting shared student perceptions of effective teaching practices. Our findings underscore the potential of STM to contextualize qualitative feedback, support fairer teaching evaluations, inform institutional decision-making, and examine the relationship between qualitative student comments and numeric SET ratings.