The roles of jitter, shimmer, and harmonic-to-noise ratio in perceptions of artificial intelligence (AI) conversational agents’ voices

Although many researchers have recently raised concerns that the design of conversational assistants may reinforce potentially harmful gender-based stereotypes, little is known about the factors that shape how people perceive conversational assistants’ voices. A previous study (Shiramizu et al., 2022, Scientific Reports, 12, 22479) demonstrated that perceptions of these synthetic voices are underpinned by dimensions reflecting valence- and dominance-related perceptions, found that the dominance dimension was strongly negatively correlated with voice pitch, but did not identify any acoustic parameters that significantly predicted valence-related perceptions. Here we reanalyzed that study’s open-access data and stimuli, finding that voices that scored higher on the perceived valence dimension tended to have higher jitter and shimmer values and lower harmonic-to-noise ratio values. Moreover, some of our analyses suggested that these effects of jitter, shimmer, and harmonic-to-noise ratio were more pronounced for perceptions of male than female voices. Collectively, our findings suggest that greater attention to the roles that jitter, shimmer, and harmonic-to-noise ratio play in social judgments of synthetic voices may be a further route through which designers might create conversational assistants that do not reinforce potentially harmful gender-based stereotypes. This research was supported by EPSRC grant EP/T023783/1, awarded to Benedict Jones, and NSERC / CRSNG grant RGPIN-2023-05146, awarded to David Feinberg. Daria Altenburg was supported by Grant BOF.24Y.2019.0006.01 of Ghent University, awarded to Adriaan Spruyt. For the purpose of Open Access, the authors have applied a Creative Commons Attribution (CC BY) to any Author Accepted Manuscript (AAM) version arising from this submission.