Unsupervised Multi-Domain Data Selection for Asr Fine-Tuning

Fine-tuning can be used to adapt an Automatic Speech Recognition system to a new domain, based on some transcribed data from the target-domain. However, in real-world settings, the availability and the amount of target-domain data needed for this fine-tuning can be limited, because of budget constraints or other reasons like privacy. In such cases, a possible approach is to automatically select candidate training data from a pre-existing pool of audio data (e.g. a mix of open-source datasets), based on a sample of the target domain. In this paper we investigate unsupervised data selection techniques for fine-tuning, under a limited budget of only one hour of training data, using a multi-source and multi-domain pool of data (7 datasets, 6k hours, various genres and styles).Our method consists in 1. extracting self-supervised model representations of both modalities (text and audio) 2. learning from these representations a domain-calibrated vector representation of what a domain is in terms of origin, genre and style and finally 3. using this representation with k-nearest neighbor search for automatic data selection. Using this method we observe an improvement of the Word Error Rate of up to about 13% (3.2 WER points) on average, compared to our baselines.