What question did this study set out to answer?

The aim is to hybridize two distinct monolingual language models without retraining them.

April 16, 2026Open Access

The Final Hybrid: Merging Two Monolingual Language Models Through Genetic Tensor Expansion Without Retraining

Key Points

The aim is to hybridize two distinct monolingual language models without retraining them.
Hybridization of two monolingual language models using genetic tensor expansion techniques.
Application of zero-padding for 2D tensors and repetition for 1D tensors.
Manual alignment of different tokenizer formats before merging.
Both original languages are preserved in the hybrid model.
No single language model dominates the output.
Emergent cross-lingual capabilities were successfully observed.

Abstract

This work documents the successful hybridization of two monolingual language models without retraining. Each model was trained on a different single language. The method uses zero-padding expansion for 2D tensors and repetition for 1D tensors. RAM is limited to 4GB. The tokenizer formats differ (tokenizer.json vs vocab.json) and require manual alignment before hybridization. Results confirm that both original languages are preserved, no single model dominates, and emergent cross-lingual capabilities are observed. The hybrid model can evolve further through additional hybridizations. Code is available on GitHub: https://github.com/mouad-tarif/Monolingual-Hybridization

Read Full Paperexternally

AI에게 질문

Bookmark

View Full Paper