Spatial Topology Traversal for Diverse Token Generation: A Non-Transformer Approach Using 3D Graph Navigation

Modern language models generate text via sequential sampling from dense probability distributions. This paper proposes an alternative inference mechanism: representing tokens as points in a learned 3D activation space, constructing a sparse neighborhood graph over those points, and traversing the graph to produce diverse, non-repetitive token sequences. Unlike autoregressive transformers, this approach leverages spatial topology rather than sequential history to determine what comes next. Key results: - 8.60% token repetition vs. 22.33% for temporal baseline (61% relative reduction) - 3,574 unique tokens across 350 prompts - 27,741-node spatial graph with k-NN edges - Extensive ablation: vector field contributes only +0.92pp; spatial neighbor query carries primary signal - Open-source implementation at https://github.com/oldnordic/spatial-topology-traversal