LlmMT+1: Enhancing Non-dominant Language Pair in Large Language Model-based Machine Translation

Large language model (LLM) based machine translation (MT) has gained increasing attention. However, most LLM-based MT models are primarily designed for dominant-language pairs and perform poorly on translation between non-dominant language pairs. We define a dominant-language pair as one where at least one language is dominant in the model’s pre-training corpus. In contrast, non-dominant pairs consist of two languages that are underrepresented or unseen during pre-training. To address this limitation, we propose a generalizable recipe, LlmMT+1. LlmMT+1 enhances non-dominant language translation by adding a new language pair to an existing LLM-based MT model, adapting the training strategy, including continual pre-training, supervised fine-tuning, and direct preference optimization based on the pre-training status of the involved languages. We apply this recipe to two types of LLM-based MT models: (1) an English-dominant model (TowerBase-13B), whose training corpus is predominantly English, and (2) a multilingual model (Aya-expanse-8B), which supports 23 languages with a more balanced multilingual training strategy. We evaluate six non-dominant language pairs based on the pre-training status of the involved languages across these two model types and demonstrate that LlmMT+1 consistently improves translation performance. Our ablation studies highlight the effectiveness of preference optimization in adapting multilingual models to new translation pairs.