Answers are grounded in this paper and cite their sources.
Mixture-of-Experts (MoE) language models hold most of their weights in experts, and quantizers calibrate those experts almost always on English text. We ask what this costs a low-resource language. We compress Qwen3-30B-A3B-Instruct-2507 for Kazakh and Russian by (i) measuring, layer by layer, which experts the router uses on Kazakh and Russian text, (ii) giving 4 bits to the experts that carry 90% of that traffic and 3 bits to the rest, and (iii) quantizing each expert with GPTQ, calibrated only on the Kazakh and Russian tokens routed to it. The result is a real file of 15.24 GB instead of 61.1 GB (4.0x smaller). Measured from the written file, perplexity on parallel Belebele passages grows by +1.3% for Kazakh, +0.4% for Russian and +3.2% for English; KazMMLU accuracy changes from 66.8% to 65.1% (Kazakh) and from 71.4% to 70.9% (Russian), differences we cannot distinguish from noise on our sample. At equal memory, choosing bits and calibration text by English instead of Kazakh and Russian raises the Kazakh perplexity loss from +0.9% to +14.3%. Against popular GGUF quantizations of the same model scored on the same exam, between 15 and 19 GB only our two files stay within 2% for both languages. We also report what does not work: removing experts erases facts, and 2-bit experts break the model although Kazakh perplexity stays within 2%. The study covers one model and one seed; code, per-question scores and notebooks that run on a free GPU are public. Code and per-question scores: https://github.com/s0nakh/linguapruneModels: kk+ru 4/3-bit, kk+ru 4-bit AI assistance: the author used Claude (Anthropic) as an additional tool; see the footnote on the first page.
No takes yet. Share an insight, caveat, or question.
Sophia Akhmetova (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: