Estimating language similarity is useful for a range of NLP tasks, including the development of multilingual language models, selecting pivot languages in machine translation, and studying cross-lingual transfer. In particular, exploiting cross-lingual transfer is crucial for low- and mid-resource languages, as it can greatly improve model performance without requiring additional data in the target language. However, choosing auxiliary languages for cross-lingual transfer is often a tedious, resource-intensive process based primarily on intuition rather than objective criteria. While some heuristics and models have been proposed for tasks such as machine translation, part-of-speech tagging, and dependency parsing, no established guidelines exist for language selection in ASR (Automatic Speech Recognition). Estimating language similarity in the ASR context is especially challenging because it involves multiple dimensions of language— phonology, syntax, and lexical overlap among them. In this talk, we explore linguistic similarity within the Romance dialectal continuum, considering both (1) a priori linguistic knowledge and (2) the internal representations of languages in text-based models (BERT) and speech models (Whisper, Wav2Vec2, HuBERT). We investigate whether these multifaceted perspectives can guide auxiliary language selection in Catalan ASR.
Barbara Scalvini (Mon,) studied this question.