Agent-Trace Fine-Tuning of Small Language Models under Constrained Compute

Small language models (SLMs, 3-8B) are increasingly fine-tuned on agent traces (recorded multi-step sessions of a frontier model using tools) in the hope of transferring agentic behavior at a fraction of the serving cost. In practice these fine-tunes are usually released with either loss-only evidence or self-reported, non-reproducible benchmark tables, and almost never with a baseline measured under the same conditions. We ask whether a single practitioner, using only individual 16 GB-class GPUs, can produce agent-trace fine-tunes that are useful and honestly characterized, and we document what it takes to get there. Our first, literature-naive recipe reduced held-out trace loss substantially on every model trained, yet regressed general coding ability by 18.9 points on HumanEval and 18.3 on HumanEval+ relative to the base model: a textbook case of catastrophic forgetting that its loss curve completely hid. We rebuild the pipeline around three evidence-backed changes: completion-only loss masking, mixed-domain replay, and fuzzy train/test decontamination with post-hoc containment measurement against both reported benchmarks. On Granite-3B, the revised recipe recovers roughly half of the lost coding ability (mean 71.7/67.3 HumanEval/HumanEval+ over three runs, vs base 81.7/76.2) and reduces to zero on our probe a distribution-mismatch failure we name and characterize as session leakage. A cross-family run on Qwen3-4B reproduces the recipe's behavior and surfaces a failure class benchmarks do not measure: a 21% empty-answer rate invisible to HumanEval and loss curves alike. Six of our headline results were artifacts of the apparatus or our own process, each caught only by re-measurement against a same-instrument baseline. We argue that at this scale the binding constraint on useful specialization is evaluation discipline, not compute, and release the full recipe, the 10.6k-trace corpus, the harness, and a complete experiment log to make the claim checkable. All resulting models, the trace corpus, and every evaluation script are released publicly as the Parable model series: https://huggingface.co/collections/AnkitAI/parable