Mixed-Cultural Speech for Intelligent Virtual Agents - the Impact of Different Non-Native Accents Using Natural or Synthetic Speech in the English Language

This paper presents an exploratory study investigating the impact of non-native accented speech on the perception of Intelligent Virtual Agents (IVAs). In an online study, native English speakers watched a video of an IVA holding a monologue whilst speaking English with either a Spanish, Hindi or Mandarin accent that was either recorded by native speakers of that respective language (natural speech) or synthetically generated (synthetic speech). The results showed a significant impact of naturalness of speech on the IVAs perceived warmth and a significant interaction of accent and naturalness of speech on its perceived competence. The naturalness of speech impacted the participants’ perception of the IVA as a non-native speaker of English, and the correctness of the attributed mother tongue in the Spanish and the Mandarin accent condition. These results are a valuable contribution to research on mixed-cultural IVAs in general and non-native speech as a cultural cue more specifically.