Beyond English-Only GRPO: Training Language and Auxiliary Reward as Implicit Regularizers in Sub-3B Math Reasoning

We study Group Relative Policy Optimization (GRPO) post-training of a 1.5B distilled reasoning model under a single-A100 LoRA constraint, comparing three arms that vary exactly one axis: training language (English vs Vietnamese-translated) and reward function (with vs without a fastText- based language-consistency reward R5). On the DeepSeek-R1-Distill-Qwen-1.5B base, English-only GRPO gains 7.5 percentage points (pp) on AMC23 but loses 16.7 pp on AIME-2024 — a benchmark-specific overfit at 50 steps. Adding the language-consistency reward R5, even when it fires uniformly at 1.0 on English training data, recovers 13.3 pp on AIME-2024 and produces the highest mean accuracy across four hard math metrics. Vietnamese-translated training data shows a milder version of the same regularization effect. Both findings hold on a $20 single-GPU LoRA budget, suggesting that auxiliary reward components and out-of-domain training data offer cheap regularization mechanisms for sub-3B GRPO. We release training/eval scripts, all checkpoints, and per-step reward traces. Code: https://github.com/nhockid235/xling-grpo-sub3b