On-Device Customization of Tiny Deep Learning Models for Keyword Spotting With Few Examples

Designing a customized KeyWord Spotting (KWS) Deep Neural Network (DNN) for tiny sensors is a time-consuming process, demanding training a new model on a remote server with a dataset of collected keywords. This paper investigates the effectiveness of a DNN-based KWS classifier that can be initialized on-device simply by recording a few examples of the target commands. At runtime, the classifier computes the distance between the DNN output and the prototypes of the recorded keywords. By experimenting with multiple TinyML models on the Google Speech Command dataset, we report an accuracy of up to 80% using only ten examples of utterances not seen during training. When deployed on a multi-core microcontroller with a power envelope of 25 mW, the most accurate ResNet15 model takes 9.7 msec to process a 1 sec speech frame, demonstrating the feasibility of on-device KWS customization for tiny devices without requiring any backpropagation-based transfer learning.