Do acoustic cues influence the McGurk effect—an audiovisual speech illusion?

Visual and auditory information are seamlessly integrated when speech is perceived. This is illustrated by the McGurk effect, where incongruent visual information from the speaker's face alters the consonant the listener hears. However, it remains unclear how acoustic features of speech influence this illusion. In this study, perceptual data from 52 listeners exposed to McGurk stimuli (acoustic [pa] with visual [ka] spoken by four speakers) were combined with acoustic analyses of [pa], [ta], and [ka] to examine whether key features of plosive consonants, the noise burst, and formant transitions influence the effect. The following six acoustic cues of these features were analyzed: the center of gravity and duration of the noise burst and the frequency range and duration of the second and third formant transitions. Linear mixed modeling showed that most of these features contributed to the illusion, with the duration of the second formant transition being the strongest cue. In some cases, perception shifted as the acoustic difference decreased; for example, the McGurk stimulus was heard more as TA when the second formant transition of [pa] resembled that of [ta]. This study demonstrated that specific acoustic phonetic cues contribute systematically to the McGurk effect.