The visualization of musical performance serves as an effective tool for performance feedback, supporting for hearing-impaired person, and inspiring artistic expression. In previous research, we proposed a system for visualizing musical fields by adapting the sound field visualization system “OTOMIRU®?.” This system consists of a 16-channel microphone array, a measurement system on PC, and a visualization system on tablet devices. Sound power levels and frequencies are measured, and these parameters are rendered onto planar objects superimposed on the real-world environment via the visualization system. Furthermore, the parameters can be freely mapped to visual attributes such as color, saturation, and brightness. However, our previous studies did not assess the extent to which the visualized musical field information is comprehensible. Therefore, in this research, we quantitatively evaluate the effectiveness of the proposed system. To achieve this, we extract features from the visualized images and investigate the extent to which they reproduce the original sounds. Specifically, given that the system visualizes sound power levels and frequencies, we extract these parameters from the visualized images and assess the discrepancy between the original sound data and the auditory information derived from the images. Additionally, we explore optimal parameter settings for enhancing the visualization fidelity.
Teraoka et al. (Wed,) studied this question.