A High-Accuracy and Ultra-Energy-Efficient Keyword Spotting Processor With Adaptive Noise Suppression and Multi-User Speaker Verification

This is the authors' abstract. We don't add key points for this paper.

In recent years, keyword spotting (KWS) processors have been proposed for offline speech recognition on edge devices. However, existing KWS processors face several challenges: 1) their accuracy is sensitive to noise, especially human voice (HV) noise in the environment; 2) insufficient exploitation of domain-specific features for energy reduction while maintaining accuracy; and 3) lack of support for multi-user speaker verification (SV) to assist in achieving user-specific KWS. To address these issues, this work introduces a KWS and SV processor (KASP) with four techniques to improve the accuracy and reduce energy consumption, including a KWS-driven adaptive direction of arrival (DoA) estimation and beamforming (BF) architecture with lightweight frequency-domain BF and adaptive DoA frequency-channel selection for energy-efficient noise suppression, a progressive wake-up processing architecture to reduce energy consumption while maintaining accuracy, and a lightweight multi-user SV technique to avoid user-specific neural network (NN) retraining and reduce energy consumption. Fabricated with 55-nm CMOS technology, the proposed KASP achieves high KWS&SV accuracy of 96.8% and 94.5%. With the adaptive noise suppression based on DoA estimation and BF, it achieves 4.2%–14.2% improvement in KWS accuracy under HV noise and non-human voice (non-HV) noise with ultralow energy consumption (1.86μJ/classification for clean audio, 2.24μJ/classification for noisy audio). Compared with the state-of-the-art designs, it can achieve up to 10.8% accuracy improvement and up to8.8×energy saving for KWS. More importantly, it is the first processor supporting KWS and multi-user SV with on-chip adaptive DoA estimation and BF.