Answers are grounded in this paper and cite their sources.
We present a new dataset called Real Acoustic Fields (RAF) that captures real acoustic room data from multiple modali-ties. The dataset includes high-quality and densely captured room impulse response data paired with multi-view images, and precise 6DoF pose tracking data for sound emitters and listeners in the rooms. We used this dataset to evaluate existing methods for novel-view acoustic synthesis and impulse re-sponse generation which previously relied on synthetic data. In our evaluation, we thoroughly assessed existing audio and audio- visual models against multiple criteria and proposed settings to enhance their performance on real-world data. We also conducted experiments to investigate the impact of incorporating visual data (i.e., images and depth) into neu-ral acoustic field models. Additionally, we demonstrated the effectiveness of a simple sim2real approach, where a model is pre-trained with simulated data and fine-tuned with sparse real-world data, resulting in significant improvements in the few-shot learning approach. RAF is the first dataset to provide densely captured room acoustic data, making it an ideal resource for researchers working on audio and audio-visual neural acoustic field modeling techniques. Demos and datasets are available on our project page.
No takes yet. Share an insight, caveat, or question.
Chen et al. (2024) studied this question.