Why is a sinusoidal signal input effective in time-domain neural vocoders?

Neural vocoders often use parametrically generated sinusoidal signals based on the fundamental frequency as input, in combination with other acoustic features such as a mel-spectrogram or a spectral envelope sequence. While multiple sinusoidal signals are often used to generate harmonic components, it is widely recognized that even a single sinusoid corresponding to the fundamental frequency can enhance both speech quality and the controllability of the fundamental frequency. This appears counterintuitive, given that natural speech contains a rich set of harmonic components extending well beyond the fundamental frequency. In this presentation, we theoretically analyze this phenomenon in the context of time-domain neural vocoders. Our analysis indicates that nonlinear operations within the neural network effectively generate harmonic components from a single-frequency input, functioning similarly to frequency modulation. These findings offer new insight into the internal signal processing behavior of time-domain neural vocoders and provide a foundation for understanding how a harmonic structure can emerge from a simple excitation signal. [Work supported in part by JST AIP Acceleration Research JPMJCR25U5, Japan, and in part by the Japan Society for the Promotion of Science (JSPS) Grants-in-Aid for Scientific Research (KAKENHI) under Grant 24KJ1236.]