CodecMOS: Singing MOS Prediction through the Integration of Self-Supervised Speech Representations and Neural Audio Codec Features

This study proposes CodecMOS, a prediction model that automatically estimates the naturalness Mean Opinion Score (MOS) of singing voices by integrating Self-Supervised Learning (SSL)-based speech representations with latent representations from a Neural Audio Codec (NAC). Although conventional MOS prediction models utilizing SSL representations are highly accurate for read speech, their performance significantly degrades for singing voices due to pitch fluctuations and greater acoustic diversity. CodecMOS aims to complement the semantic information captured by SSL with the acoustic information encoded in the latent representations of NAC. Experimental evaluations comparing several fusion strategies demonstrate that the model fusing wav2vec 2.0 and Descript Audio Codec features via Feature-wise Linear Modulation (FiLM) achieves the highest correlation coefficients, outperforming existing methods. This work is the first to systematically investigate the complementary relationship between SSL and NAC representations in predicting the naturalness of singing voices.