The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Convergent Category Geometry in Small Language Models

Bürger et al. (2024) demonstrated that truth representations in large language models areuniversal across statement polarity but reside within a multidimensional subspace. We extendthis framework along three questions: how the dimensionality of the subspace depends on themodel’s knowledge, which architectural component builds the truth direction, and what thedirection is a mixture of. In Part I, a training-free directional probe derived from the SVD ofhidden-state minimal pairs shows that the dimensionality of truth is knowledge-dependent: thesignal concentrates on a single axis for known facts and diffuses as knowledge decreases. InPart II, a relational law emerges across multiple model families: attention propagates truthframes, the feed-forward network opposes the current block’s frame, and post-peak decay iscausally attributed to the SwiGLU value stream. Furthermore, per-category truth axes forma semantically signed arrangement that converges across families. Stress tests expose a signinstability in this orientation, which we repair with a spectral consensus gauge to sharpenthe convergence into a knowledge-gated law. Finally, a replication campaign on Gemma-2-2b,extending our decomposition tools to accommodate its sandwich normalization, confirms theselaws and attributions. We quantify the knowledge gate as classical attenuation and isolate astable, model-specific private geometry.