RoPE Is a Substrate, Not a Trick: Structural Separation between Toroidal Composition and Transformer Primitives on Finite Groups

This work explains a persistent empirical puzzle: why Rotary Positional Embeddings (RoPE) enable transformers to succeed on certain compositional reasoning tasks where standard architectures fail. We show that the limitation is not merely empirical but structural. Additive residual updates in Euclidean space cannot represent non-trivial finite group actions, regardless of depth, width, or training budget. The obstruction is geometric: Euclidean space is torsion-free, whereas finite groups are not.This yields a sharp separation between composition mechanisms. Purely additive layers fail at the first non-trivial composition step. Generic MLPs exhibit a positive error drift that accumulates with depth. In contrast, RoPE implements a representation equivalent to a one-dimensional torus (T¹), enabling stable composition at machine precision. Extending this construction, we show that full toroidal substrates (Tⁿ) support stable composition over at least 10¹⁵ steps with linear cost per step.We validate these predictions on iterated modular arithmetic and group-composition tasks. A real instruction-tuned model (Qwen2.5-0.5B) achieves partial accuracy on shortcut-solvable instances but fails completely (0/60) on state-dependent tasks without closed-form reductions. The toroidal substrate, by contrast, achieves perfect accuracy (60/60) at O(d) cost. Finally, we show that gradient descent on a contrastive objective recovers the correct group homomorphism without supervision, converging to machine precision.These results suggest that RoPE's success is not an engineering artifact, but a manifestation of an underlying toroidal geometry required for stable compositional reasoning.