What does this research mean for the field?

The proposed hybrid approach for translating Maghrebi dialects into/from Modern Standard Arabic using a transformer architecture with BERT embeddings significantly improves translation performance compared to existing models. Novelty: ClaimNovelty.NOVEL_FINDING. Consensus alignment: ConsensusAlignment.NEUTRAL.

What question did this study set out to answer?

The aim is to improve translation of Maghrebi dialects by leveraging transformer architectures and transfer learning techniques.

March 12, 2026Open Access

Maghrebi dialects – Arabic bidirectional translation: an improved transformer with transfer learning

JRJihad R’baitiUniversité Mohammed VI Polytechnique YHYoussef HmamoucheUniversité Mohammed VI Polytechnique ASAmal El Fallah Seghrouchni

Key Points

The aim is to improve translation of Maghrebi dialects by leveraging transformer architectures and transfer learning techniques.
Developed a hybrid translation approach using transformer and BERT models.
Incorporated BERT embeddings into the transformer’s encoder and decoder.
Trained the BERT model using Maghrebi dialect and Arabic corpora.
Conducted evaluations with various tokenisation techniques and human assessments.
Achieved BLEU score of 14.148 and a BERTScore of 79.414 for translation accuracy.
Results showed competitive performance against leading language models like ChatGPT.
Ablation studies confirmed the effectiveness of the proposed methods.

Abstract

Abstract Neural Machine Translation (NMT), a subfield of Natural Language Processing, has seen significant advancements with the emergence of transformer architectures and generative artificial intelligence, demonstrating remarkable performance in various languages. However, translating Arabic dialects remains a notable challenge that becomes very pronounced primarily due to their morphological complexity and divergence from standardised grammatical rules. In this paper, we present a hybrid approach for translating the Maghrebi dialects into/from Modern Standard Arabic (MSA). The approach takes advantage of the strengths of the transformer architecture and the BERT language model for transfer learning of representations. To achieve this, we incorporated BERT embeddings into the encoder and decoder stacks of the transformer architecture. The BERT architecture, which we utilised, was trained in a self-supervised manner on Maghrebi dialects and Arabic corpora. The resulting BLEU/BERTScore/ChrF/METEOR scores for the approach were 14.148/79.414/28.885/28.428 and 8.961/20.994/19.465 (BLEU/ChrF/METEOR) for the translation in both directions using the raw data, demonstrating competitive performance compared to ChatGPT and Gemini Large Language Models (LLMs). Furthermore, we evaluated the approach using an ablation study with fine-tuned NLLB-200 and against three combinations of tokeniser techniques used in conjunction with the transformer architecture: Byte-Pair Encoding (BPE) tokeniser, WordPiece tokeniser, and BERT tokeniser. Both evaluations, including human evaluation, confirm the efficacy of our method.

Ask AI

Helpful

Bookmark

View Full Paper