Quantization, Not Just Scale: Reliability and Failure Anatomy of Small Open-Weight Tool Agents

This preprint studies the reliability of small open-weight language-model agents (1–7B parameters) run locally at zero cost. Using a five-tool agent harness and a 25-task benchmark spanning single-tool use, multi-tool chains, distractor tools, underspecified requests, and injected tool faults, we run 2,400 seeded episodes across a model-size × quantization × mitigation matrix and grade them programmatically (blind-validated against hand labels, Cohen's κ = 0.86/0.94). We find that 8-bit quantization more than doubles agent task success over the default 4-bit build at 1.5B parameters (+16 points, p = 2×10⁻⁵) and changes how the model fails, not just how often; that a deterministic verification-and-retry guardrail does not improve overall success and can reduce it below a model-capability threshold, while still tripling recovery from tool faults at 3B; and that reliability collapses under repetition. All code, raw logs, analysis, and figures are public and reproduce every reported number: