End-to-End Speech Hash Retrieval Algorithm based on Speech Content and Pre-training

Traditional speech retrieval tasks, such as audio fingerprinting and Spoken Word Detection Query (STD-QbE), focus on feature matching for speech or keyword retrieval. In this paper, we present a content-based speech retrieval algorithm. The algorithm allows matching based on the complete content of sentences in speech, not just local features or keywords. Importantly, it completely bypasses the Automatic Speech Recognition (ASR) transcription process by mapping the acoustic features of the sentence directly to the Hamming space. Retrieval of the same content is then achieved by comparing Hamming distances, thus effectively eliminating the potential impact of transcription errors on retrieval performance. In order to achieve this, our approach employs the Connectionist Temporal Classification (CTC) speech recognition technique to pre-train the model to learn content-dependent representations of speech features. Through experiments, we demonstrate that our approach achieves excellent performance in speech retrieval tasks.