What question did this study set out to answer?

This research aims to validate the performance of the GEARS Pinion model for autoregressive language tasks using a graph-navigation approach.

April 21, 2026Open Access

GEARS Pinion: Experimental Validation of a Graph-Navigation Language Model at 2.3M Parameters

Key Points

This research aims to validate the performance of the GEARS Pinion model for autoregressive language tasks using a graph-navigation approach.
Experimental validation of GEARS Pinion with 2.29M parameters using Wikitext-2 for training.
Comparison of model performance against a vanilla Transformer baseline with similar parameter counts.
Ablation analysis to identify contributions of different model components to perplexity.
GEARS Pinion achieves a perplexity of 148.55, outperforming the Transformer baseline of 150.34 PPL.
Top-1 accuracy for GEARS Pinion is 20.79%, while the baseline achieves 19.35%.
Progressive dimensionality contributed positively to performance with a ΔPPL of +50.41.

Abstract

We report experimental validation of GEARS (Geometric Expansion with AdaptiveRouting and Scaffolds), an attention-free architecture that replaces transformerself-attention with navigation over a geometrically structured concept graph ofprogressively growing dimensionality (3D → 6D → 12D → 24D). The validated model,GEARS Pinion, has 2.29M parameters and is trained for autoregressive languagemodeling on Wikitext-2 with an 8,192-token BPE vocabulary. On the test set, GEARSPinion reaches a perplexity of 148.55, outperforming a parameter-matched vanillaTransformer baseline (150.34 PPL, 2.05M parameters) on all metrics: Top-1 accuracy20.79% vs. 19.35% (+1.44 pp), Top-5 accuracy 39.12% vs. 37.67% (+1.45 pp),LAMBADA-style accuracy 20.70% vs. 22.27%. To the best of our knowledge, this isthe first report of autoregressive language modeling results on a standardbenchmark for a graph-navigation architecture. Ablation analysis confirms thatprogressive dimensionality (vertical descent) contributes a substantial ΔPPL of+50.41, while auxiliary components (wormholes, Ricci curvature, compatibilitygraph, morphological profiles) show no measurable contribution at this scale onEnglish-only data - a result consistent with their design purpose. A controlexperiment - a pure 5-layer Mamba SSM stack with matched parameter budgetreaches PPL 123.61, indicating that at 2.3M parameters the SSM componentdominates the observed behavior and that the scale hypothesis remains thecentral open question for future work. This report is a companion to the GEARS v2.1 architecture specification(DOI: 10.5281/zenodo.19036971).

GEARS Pinion: Experimental Validation of a Graph-Navigation Language Model at 2.3M Parameters

Key Points

Abstract

Cite This Study