Bias intervention messaging in student evaluations of teaching: the role of gendered perceptions of bias

Many studies have documented discrepancies in student evaluation of teaching ratings between male and female instructors and between ethnic majority and minority instructors. Given the importance of such ratings to academic careers and the likelihood of potential intergroup bias, it is crucial that institutions consider approaches to mitigate such biases. Several recent studies have found that simple bias mitigation messaging can be effective in reducing gender and other biases. In the present research, students enrolled in several large Faculty of Science undergraduate courses at an Australian university were recruited on a volunteer basis via the course learning management system. Half of the participants were randomly assigned an intervention message highlighting potential biases relating to gender and language background. Data from 185 respondents were analysed using Bayesian ordinal regression models assessing the impact of message exposure on evaluation scores. Reading a bias intervention message caused students to significantly adjust their scores, with the nature of that change dependent on student and instructor characteristics. Among male students, the bias intervention message significantly increased scores for all except male instructors with English speaking backgrounds, for whom there was no significant impact of the message. In contrast, among female students, the bias intervention message significantly decreased scores for male instructors with English speaking backgrounds only. The sample showed an overall decrease in scores in the intervention group relative to the control group. This is the first study to detect a negative impact of bias intervention messaging on SET scores. Our results suggest students may not acknowledge their own potential bias towards instructors with whom they share similar demographic backgrounds. In conclusion, bias intervention messaging may be a simple method of mitigating bias, but it may lead to consequences in which one or more groups receive lower ratings as a result of the correction.