Lattice-Free Open Vocabulary Keyword Spotting

Keyword Spotting (KWS) systems traditionally rely on predefined keywords, yet the ability to recognize keywords in open-vocabulary contexts is crucial for enhancing personalized interactions with smart devices. Not only detecting the keyword's presence but pinpointing the location is vital. Typical statistical methods use a speech recognition system to localize the keyword. But, extracting the lattices and searching the keywords is computationally intensive. Due to the time-consuming lattice generation, these approaches are unsuitable for realtime application. This paper introduces a lattice-free KWS method that accurately detects and locates user-defined keywords. It accepts audio and text query as input, generating acoustic and keyword embeddings from acoustic and text encoders. By incorporating a Cross-attention layer, the model effectively focuses on relevant acoustic keyword features while disregarding irrelevant information. The resulting representations are then processed by a keyword localizer module for precise location prediction, while a keyword detector module estimates the keyword presence. This approach eliminates the need for lattice extraction and searching. The results suggest that this approach has robust generalization capabilities regarding domain mismatch and speaker variability. Additionally, the model demonstrates proficiency in localizing the repetitive keywords within the audio and can accurately predict the positions of keyword phrases.