This article develops a mathematically explicit, architecture-agnostic framework for semantic phase transitions in learned representations. The core idea is to model “semantic units” (concepts, features, internal symbols) as balls in a metric measure space, called an observation geometry, and to study how their overlaps, density, and quality control capacity and emergent behaviour. On the static side, the paper introduces a semantic Hamiltonian with a chemical-potential term and a convex overlap penalty. Under only Ahlfors-type regularity (no exact homogeneity), it proves a capacity inequality linking semantic count, average overlap, and an energy budget, and derives an interaction-limited scaling lawNmax≲B1/ (2−γ) N_ B^1/ (2-) Nmax≲B1/ (2−γ) for small-overlap potentials U (o) ≳oγU (o) o^ (o) ≳oγ. This yields universal constraints on admissible semantic-capacity exponents, independent of specific architectures or optimisers. On the probabilistic side, the work analyses a one-dimensional Poisson interval model, obtaining an exact cluster-density formula κ (λ) =λe−2λr () = e^-2 rκ (λ) =λe−2λr with a critical density λc=1/ (2r) c = 1/ (2r) λc=1/ (2r) separating dilute, critical, and condensed regimes. In higher dimensions, semantic centres are modelled by Poisson Boolean models and random geometric graphs, and known continuum-percolation and sharp-threshold results are used to define and constrain task events such as coverage, connectivity, and bounded-depth compositional paths. Finally, the paper proposes a mapping from geometric/energetic parameters (λ, r, p, μ, β, γ) (, r, p, , , ) (λ, r, p, μ, β, γ) to model size, effective intrinsic dimension, compute, and data quality, leading to semi-quantitative, falsifiable constraints on scaling exponents and emergent-task thresholds. The resulting framework is intended as a universal low-level reference model that any detailed theory of neural scaling laws and emergent abilities must respect.
Takahashi K. (Fri,) studied this question.