The Hardware Basin: Why the Quantization Cliff Is About Level Allocation, Not Bit Count

Paper 7 reported a universal quantization cliff at INT4→INT3 using bitsandbytes software quantization. We investigated whether this cliff is a software artifact or a mathematical property of low-precision arithmetic, and found the answer depends on the quantization method. Under symmetric uniform quantization the cliff is at INT8→INT4 (Pythia-410M and 1.4B both collapse to BPT ≈ 17 at INT4 vs. 4.3 at FP16); under NF4, which places levels at the quantiles of a normal distribution, INT4 is operational (BPT ≈ 3.9–4.7). Same bit count, opposite outcomes. The difference is level allocation: symmetric quantization wastes resolution in the sparse tails, while NF4 concentrates levels near zero where weights cluster. A Lloyd–Max (minimum-MSE) quantizer achieves cosine similarity 0.990 at INT4, consistent across four models and all 24 layers of Pythia-410M. Publication-grade verifications confirm the picture: the structural bonus is preserved by NF4 but destroyed by symmetric INT4 (Cohen's d = 400.81), and per-matrix cosine similarity overstates end-to-end quality—Lloyd–Max INT3 fails when propagated through all 24 layers. The cliff is a property of level allocation, not bit count.