VDR Toy LLM: Exact Rational Arithmetic in a Transformer Language Model Using Fixed-Denominator VDR Triples

We present a complete transformer language model — embedding, self-attention, feed-forward network, backpropagation, weight updates, and text generation — running entirely in exact rational arithmetic with no floating-point operations. The model uses the vdr-math Python library, which represents every value as an ordered triple V, D, R (Value, Denominator, Remainder) with a fixed denominator D = 2³2. We describe the denominator growth problem that arises when standard rational arithmetic operators are applied in a fixed-frame system, the operator-level solution we implemented to prevent it, and the quadratic softmax surrogate that eliminates transcendental functions from the forward pass. The resulting toy model — 181 parameters, vocabulary of 5, trained for 20 epochs on a 6-word corpus — passes 9 verification tests including bit-identical determinism across runs and exact sum-to-one softmax outputs. All denominators remain at 2³2 through every operation chain. The work demonstrates that a fixed-denominator rational arithmetic system can support a complete LLM training and inference pipeline, and identifies the engineering steps required to scale beyond toy dimensions.