Lateral position discrimination using wavefield synthesis in an open environment to test the effect of observer-controlled motion on the ventriloquist effect

We studied the ventriloquist effect, in which sound locations are perceived near visual objects, under naturalistic conditions wherein observers moved real objects in an open environment. A wavefield synthesis array produced sound sources that were either collocated with the object or offset to its left or right. Using motion tracking cameras, the sound sources could either move along with the objects while observers moved them (congruent motion) or remain stationary while the object was moved (incongruent motion). First, a 200-ms 79 dB SPL broadband sound was presented on either side of a stationary object. Twenty seated listeners discriminated the relative position of the sounds in a 2IFC task in a three-down one-up adaptive track. As expected from the ventriloquist effect, lateralization threshold was higher when the object was present than when it was absent. Next, we tested the hypothesis that auditory and visual motion congruence would promote the binding of sounds and objects, while incongruent motion would disrupt it. Lateralization thresholds were higher in the congruent motion condition in which the sound was collocated with the moving object. However, inconsistent with our predictions, thresholds were just as high in the incongruent motion condition. [Work supported by ASA SURIEA, CMU Dietrich Grant.]