The timing of an avatar’s beat gestures biases lexical stress perception in vocoded speech

Cochlear implants (CIs) are neural prostheses that restore some level of hearing capacity, albeitconveying a less fine-grained speech signal than normal hearing conditions. For example, CIsconvey altered fundamental frequency (F0) information, disrupting the perception of lexical stress(e.g., distinguishing between the noun CONtent and the adjective conTENT) in languages in whichthis feature rests on F0 modulations. CI-users can compensate for the degraded nature of theacoustic input by exploiting the audiovisual affordances of human communication, weighing moreheavily the visual information provided by the speaker (e.g., lip movements and gestures). Recentstudies showed that, in individuals with normal hearing, the timing of simple up-and-downmovements of the hand (i.e., beat gestures) biases lexical stress perception. The present study testedif the timing of beat gestures produced by an avatar can bias Dutch lexical stress perception invocoded speech, which simulates CI-hearing conditions. The bias induced by beat gestures invocoded speech was particularly pronounced when hearing an ambiguous or the least frequentstress pattern in Dutch. These results suggest that (even artificially generated) beat gestures can beused to support speech perception in CI-mediated speech, especially when processing words withprosodic features that are less frequent.