Hey AI: Can you trigger me? A comparison of Text-to-Speech synthesis and human speech in a virtual social stress paradigm

– While the use of text-to-speech synthesis (TTS) is high, evidence of its effects on presence in VR, for example, is scarce. It also remains unclear, whether virtual interactions using TTS can provoke psychosocial stress and evaluative threat as required for psychological paradigms like the Trier Social Stress Test (TSST) as the experimental gold standard to examine human stress reaction. Using TTS instead of prerecorded human speech (PHS) to auralize the agents in the virtual version (VR-TSST) can be beneficial, e.g. by creating greater freedom and flexibility to react. In a randomized-controlled trial, we compared TTS stimuli to PHS within a VR-TSST. Participants’ heart rate, trapezius muscle activity, and self-reports on stress, presence, and affective state were surveyed. In both speech conditions profound stress reactions were provoked but no significant modulations by audio were observed. Equivalency testing demonstrated equivalent heart rate increases between audio conditions but only moderate evidence for equivalency of stress and presence ratings was observed. The current findings show that social stress reactions can be provoked with synthetic speech stimuli and point towards comparable effects between TTS and PHS in socio-emotional experiences. This highlights the potential of TTS for use in virtual social interactions.