Replicating outdoor environments using VR and ambisonics: a methodology for accurate audio-visual recording, processing and reproduction

Abstract This paper introduces a methodology tailored to capture, post-process, and replicate audio-visual data of outdoor environments (urban or natural) for VR experiments carried out within a controlled laboratory environment. The methodology consists of 360 ^ ∘ video and higher order ambisonic (HOA) field recordings and subsequent calibrated spatial sound reproduction with a spherical loudspeaker array and video played back via a head-mounted display using a game engine and a graphical user interface for a perceptual experimental questionnaire. Attention was given to the equalisation and calibration of the ambisonic microphone and to the design of different ambisonic decoders. A listening experiment was conducted to evaluate four different decoders (one 2D first-order ambisonic decoder and three 3D third-order decoders) by asking participants to rate the relative (perceived) realism of recorded outdoor soundscapes reproduced with these decoders. The results showed that the third-order decoders were ranked as more realistic.