Investigates ternary compression effects on reasoning performance in models, suggesting specific implications for quantization strategy.
Post-training quantization (PTQ) of LLM weights to ternary values {-1,0,+1} has recently reached near-lossless accuracy by stacking K trit-planes, most directly in PTQTP. But how much ternary compression a model tolerates is not uniform across what the model does. We present a capability-stratified study of K-trit-plane PTQ across 22 open checkpoints—18 dense (0.5–14B, seven families, base and instruct variants) and four mixtures-of-experts (7–30B total parameters)—evaluated with the official lm-evaluation-harness. Three findings. (1) K is the lever: K=3 (≈ 4.9 bpw) recovers ≈baseline almost everywhere (90% mean GSM8K, 97% knowledge), while K=2 (≈ 3.3 bpw) collapses reasoning specifically—grade-school math to a near-zero cliff and competition math to ≤ 31%—yet leaves knowledge, commonsense, and code largely intact (38–95%). The cliff is a property of uniform extreme quantization, not of ternary weights per se: a controlled per-bit head-to-head (Qwen2.5-1.5B, N=200) shows a uniform 3-bit integer (RTN) collapses reasoning just as ternary K=2 does (both ≤ 2% GSM8K at ≈3–4 bpw), while the mixed-precision integer K-quants that survive (Q2_K) do so by keeping reasoning-critical layers at higher precision. The lever is thus a clean, graded instance of a low-bit reasoning cliff, not a ternary-only phenomenon. (2) The collapse is functional, not a weight-reconstruction failure: it lives in the layers—late-activation SQNR is lower for math than for knowledge in 12/12 models—and the full-precision/ternary divergence is frontal, not cumulative. (3) Where to spend the extra bits is mostly open: attention is the causal bottleneck (protecting it is necessary, 4/4 families), but cheap per-block allocation signals are not robustly better than random. Read the other way, K is an instrument, not merely a compression knob: a graded, reversible, weight-level dial that selectively ablates multi-step reasoning while leaving stored knowledge intact—unusually graded and reversible for a compression knob (a uniform integer bit-width toggles the same collapse only coarsely: INT3 off, INT4 on), a model organism for reasoning under compression. We do not propose a new quantizer: a handful of recent methods already reach near-lossless ternary PTQ (mostly unreviewed preprints, only one with released code), and PTQTP already reports a capability breakdown—but none gives the mechanism, the uniform-vs-mixed control that locates the cliff, or the allocation analysis. We map the K-lever, its mechanism, and its limits, and recommend K=3 as a uniform, near-universal operating point (competition math is the one capability still gaining marginally from K≥4). And because the collapse is invisible to perplexity and aggregate accuracy, it sharpens an established warning: audits of compressed models should be capability-stratified. All raw logs, the harness configuration, and the figure scripts are released.
No takes yet. Share an insight, caveat, or question.
Adriel Domingues de Souza Andrade (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: