Characterizes statistical organization in semantic embedding spaces, suggesting new insights into language models.
Semantic embedding spaces are the numerical substrate of modern language models, yet their global statistical organization has never been characterized as a law. We report Ryan JHANG’s Law of Semantic Gravitation: when semantic states self-organize under competing tendencies, they settle into a statistical equilibrium exhibiting four universal statistical invariants — statistical equilibrium, heavy-tailed structure, effective- dimension reduction, and multi-scale fractality. Strikingly, the effective intrinsic dimension stabilizes at d ≈ 8 regardless of the ambient embedding dimension (testedacross 20–50 dimensions and multiple frequency bands), while cluster densities vary freely — exactly as a law of invariants predicts. We further show, via scale-invariance, that these power laws share a common mathematical origin with Neural Scaling Laws, and provide a conditional-coupling analysis of their causal relationship. This work anchors the phenomenological law and its invariants; the generating process is reserved for a subsequent release.
No takes yet. Share an insight, caveat, or question.
Rong Jhou Jhang (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: