Answers are grounded in this paper and cite their sources.
Post-training methods are usually studied inside large frameworks, on real language models, where a wrong gradient or a lucky seed is hard to see. This study rebuilds three of them from scratch in NumPy, LoRA, GRPO, and DPO, on synthetic tasks small enough to check by hand, and then surrounds every headline number with a 10-seed study: full fine-tuning and rank baselines, a KL beta sweep, ablations, a 36-setting DPO grid, and retention checks with aligned, misaligned, and shared-weight preferences. The study was not pre-registered: the scope document was committed on 4 November 2026 before any code, but no hypothesis or pass line was written before a run, so every hypothesis below is post hoc and scored as descriptive. The headline result is the rigor study's SFT comparison: a rank 2 LoRA adapter (40 trainable weights) reaches 10-seed mean held-out test accuracy 0.9039 (bootstrap 95% CI 0.8742 to 0.9305), below full fine-tuning of all 64 weights at 0.9391 (CI 0.9219 to 0.9555), on a task whose target change is rank 2 by construction, the case most favourable to the adapter. GRPO's KL term trades reward for closeness to the reference exactly as its loss says it should (at 60 iterations, beta 0 gives mean reward 0.9637 at KL 1.1980; beta 0.1 gives 0.8859 at KL 0.9544). DPO on checker-aligned preferences generalises to unseen rejected contrasts (held-out preference accuracy 1.0000 on all 10 seeds), and DPO on preferences that contradict the task removes the task exactly where they disagree (greedy accuracy 0.0000 on misaligned prompts), with the damage spreading to unseen prompts once weights are shared (0.5375 aligned against 0.2500 with 4 of 8 train prompts misaligned). Three recorded numbers do not survive their own seed study unchanged: the recorded GRPO reward of 0.9766 is the best of 10 seeds (mean 0.9465, CI 0.9340 to 0.9590), the recorded DPO held-out margin of 0.8297 is the lowest of 10 seeds (mean 0.8832), and the Build GRPO gradient does not match the loss it reports, a defect found by finite differences, kept on the record by a written decision, and quantified by a corrected-gradient ablation (mean reward 0.9180, CI 0.9027 to 0.9332). A reward hacking probe shows the same pattern in miniature: a policy that only suppresses rejected actions games the DPO train contrast to 1.0000 while held-out preference accuracy stays 0.4375. No real language model was trained anywhere in this study, and no number here is claimed to transfer to one.
No takes yet. Share an insight, caveat, or question.
Jesun Ahmad Ushno (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: