TetraNet: Power-of-Two Quaternary Weights for Small Decoder-Only Language Models

Ternary language models offer an attractive, multiply-free representation for efficient infer-ence, but inherently leave a portion of a standard two-bit hardware register unutilized. Westudy whether this unused state can be reclaimed to improve model capacity without sacrificingarithmetic efficiency. TetraNet introduces a symmetric quaternary weight grid {−1, −c, c, 1}.By restricting the intermediate states to powers of two (c ∈ {1/4, 1/2}), the core dot productcan be evaluated using only signed integer additions and bit-shifts, entirely bypassing hardwaremultipliers. In a controlled pilot study using ∼10M-parameter decoders trained on TinyStories,both quaternary variants consistently improve validation perplexity over a matched ternarybaseline. Furthermore, analytical hardware modeling indicates that TetraNet achieves totalenergy efficiency comparable to idealized ternary networks. By fully saturating the two-bitmemory bandwidth, the model yields an approximate 6× energy reduction over full-precisionarchitectures. We frame this work as an exploratory representation design point, demonstratingthe viability of shift-based quaternary networks prior to large-scale deployment.