Agentic Weight-Space Programming as an Alternative to Fine-Tuning in Small Language Models

Small reasoning language models often fail by looping: with thinking on and a 12,288-token budget, Qwen3.5-0.8B runs out of space before answering on about 85% of grade-school math problems. We ask whether frontier-model agents can repair this defect by editing the weights directly, without gradients, and how the result compares with simpler fixes. Claude agents, given the weights, a small testing budget and at most four rank-1 edits, found fixes in about two hours per run on a single consumer GPU (GTX 1660 SUPER, 6 GB). The edits raise accuracy on locked GSM8K test problems from 15.5% to 61.5–68% (two locked sets of 200, the second with every arm and prediction registered before opening), and six runs of the same protocol all reached 58–67% on practice problems. Independent agents found the same “Wait”-brake neurons and the same lever. The edit that GPT-6 Astra wrote in Sparks of Weight-Space Programming (Takehi et al., MIT CSAIL, 2026) reaches the same accuracy only with a 20,480-token budget, so the agents’ advantage over it is efficiency: 1.7–1.9× fewer tokens. The agents’ edits also beat a LoRA fine-tuning baseline that used 13× more GPU time. Simple alternatives are strong: over the 400 locked problems, switching thinking off is only 6.5–9 points behind the edits at a third of their tokens, and ties the edits alone on MATH-500. A scripted search with the same tools but no agent reasoning failed, every cheap proxy score was eventually gamed, and more interpretability context hurt. On a diffuse defect (instruction following, IFEval), neither the agents, nor prompting, nor fine-tuning helped. Every prediction was written in a dated lab notebook before the result. The PDF is provided in English and in Brazilian Portuguese.