Visual and auditory information are seamlessly integrated when speech is perceived. This is illustrated by the McGurk effect, where incongruent visual information from the speaker's face alters the consonant the listener hears. However, it remains unclear how acoustic features of speech influence this illusion. In this study, perceptual data from 52 listeners exposed to McGurk stimuli (acoustic pa with visual ka spoken by four speakers) were combined with acoustic analyses of pa, ta, and ka to examine whether key features of plosive consonants, the noise burst, and formant transitions influence the effect. The following six acoustic cues of these features were analyzed: the center of gravity and duration of the noise burst and the frequency range and duration of the second and third formant transitions. Linear mixed modeling showed that most of these features contributed to the illusion, with the duration of the second formant transition being the strongest cue. In some cases, perception shifted as the acoustic difference decreased; for example, the McGurk stimulus was heard more as TA when the second formant transition of pa resembled that of ta. This study demonstrated that specific acoustic phonetic cues contribute systematically to the McGurk effect.
Moisio et al. (Fri,) studied this question.