JLMS25 and Jiao-Liao Mandarin Speech Recognition Based on Multi-Dialect Knowledge Transfer

This is the authors' abstract. We don't add key points for this paper.

Jiao-Liao Mandarin, a distinguished dialect in China, reflects the linguistic features and cultural heritage of the Jiao-Liao region. However, the labor-intensive and costly nature of manual transcription limits the scale of transcribed corpora, posing challenges for speech recognition. We present JLMS25, a transcribed corpus for Jiao-Liao Mandarin, alongside a novel multi-dialect knowledge transfer (MDKT) framework for low-resource speech recognition. By leveraging phonetic and linguistic knowledge from neighboring dialects, the MDKT framework improves recognition in resource-constrained settings. It comprises an acoustic feature extractor, a dialect feature extractor, and two modules—WFAdapter (weight decomposition adapter) and AttAdapter (attention-based adapter)—to enhance adaptability and mitigate overfitting. The training involves a three-phase strategy: multi-dialect AID-ASR multi-task learning in phase one, freezing the dialect feature extractor in phase two, and fine-tuning only the adapters in phase three. Experiments on the Jiao-Liao Mandarin subset of the KeSpeech dataset and JLMS25 dataset show that MDKT outperforms full-parameter fine-tuning, reducing Character Error Rate (CER) by 5.4% and 7.7% and Word Error Rate (WER) by 6.1% and 10.8%, respectively.