What question did this study set out to answer?

This research aims to enhance audiovisual congruency in head-mounted displays (HMD) by investigating front-of-ear sound reproduction methods.

June 13, 2026Open Access

A study on improving audiovisual congruency in HMD systems through sound externalization based on front-of-ear speakers and crosstalk transfer functions

Key Points

This research aims to enhance audiovisual congruency in head-mounted displays (HMD) by investigating front-of-ear sound reproduction methods.
Compared in-ear, off-ear, front-of-ear, and CTF-applied front-of-ear audio conditions.
Evaluated the role of visual information on sound perception.
Utilized real-time DSP for crosstalk transfer function processing.
In-ear condition showed significant sound internalization.
CTF-applied front-of-ear condition demonstrated enhanced sound externalization and frontal orientation.
Visual information supported auditory stability but played a limited role.

Abstract

This study proposes a front-of-ear–based acoustic reproduction approach that combines a front-of-ear speaker configuration with a crosstalk transfer function (CTF) to address the issues of sound internalization and audiovisual incongruence that are commonly observed in conventional in-ear–based audio presentation methods within HMD-based virtual reality environments, and examines its perceptual effects empirically. The proposed system is designed to facilitate frontal sound externalization that is congruent with visual information in HMD usage contexts by approximating, via real-time DSP-based CTF processing, interaural time differences (ITDs), interaural level differences (ILDs), and acoustic transmission characteristics shaped by the head and pinnae that are observed in natural listening environments. In the experiment, in-ear, off-ear, front-of-ear, and CTF-applied front-of-ear conditions were compared, and the effects of visual information availability were additionally evaluated. The results showed that sound internalization was salient in the in-ear condition, whereas off-ear and simple front-of-ear conditions were associated with partial externalization but exhibited limitations in terms of directional stability and frontal localization. In contrast, the CTF-applied front-of-ear condition was associated with perceived sound externalization, frontal orientation, and perceived distance, and visual information appeared to play a supportive role in stabilizing these auditory percepts. Overall, this study suggests that frontal sound externalization in HMD environments can be supported through auditory reproduction approaches and signal processing, while visual information appears to play a more limited, supportive role. These findings offer insights into the design of auditory reproduction systems for HMD-based virtual reality environments, particularly with respect to frontal-oriented sound externalization.

Perguntar à IA

Bookmark

View Full Paper