Frequency-Aware Multi-Rate Resampling with Multi-Band Deep Supervision for Modular Speech Denoising

Conventional waveform-based speech enhancement models prioritize temporal modeling, often neglecting the irreversible spectral information loss triggered by standard downsampling. Consequently, this study introduces a novel frequency-aware framework. The proposed approach incorporates a modular, multi-rate resampling module with principled anti-aliasing to precisely control each layer’s effective frequency band, complemented by a multi-band loss function for deep supervision. Integrating this module into a standard Wave-U-Net and an attention-enhanced variant confirmed its effectiveness. The findings show a significant improvement over the baseline, yielding an average Perceptual Evaluation of Speech Quality gain of 0.40, with further benefits when paired with an advanced temporal model at a permissible increase in computational complexity. Furthermore, tests on novel noise types validate the generalizability of the proposed principles, establishing structured frequency band allocation as a fundamental, modular design strategy for improving end-to-end models.