Towards Understanding the Entanglement of Human Stereotypes and System Biases in Human-Robot Interaction

The reproduction of stereotypes and social biases are critical issues in Artificial Intelligence research. Current research focuses mainly on identifying and minimizing biases in systems. Less research has been done on the interplay between system biases and stereotypes in humans and their social effects, such as automation bias and stereotype threat. In this paper, we want to bring attention to these topics in the domain of human--robot interaction. In particular, we analyze possible influences on automation bias in a dataset from an empirical human--robot interaction study. We observe automation bias when participants believe a Furhat robot's false judgment of their language skills to be accurate. Despite the limited data, we find that being bilingual significantly influences participants' belief in the robot's negative assessment of their language skills. This result shows that participants' insecurity about their own (language) skills can be reinforced by automation bias and vice versa. We illustrate and discuss the need for awareness of automation bias and the possible reinforcement of this effect due to other social biases.