Develops a comprehensive model for geometric phase transitions in transformer architectures, indicating new insights into information processing.
When a transformer processes text, its internal representations pass through a sequence of layers, each performing a different kind of geometric computation. Geometric Contrast Imaging (GCI) is a measurement method that classifies the computational geometry at each layer by analyzing patterns in how attention heads route information. The resulting "geometric trace" reveals that inference is not a uniform process: models consistently organize their layers into distinct geometric phases, with structured transitions between them. This paper asks: what forces govern how the geometric character of computation changes from one layer to the next? We present a five-force Lagrangian equation family that models the acceleration of geometric change through depth. The five forces are: (1) geometry-curvature coupling, which accelerates computation when the geometric mode changes; (2) exponential information pressure, which increases toward the output layer as the model commits to a prediction; (3) spherical-phase oscillation, capturing periodic modulation during integrative processing; (4) phase boundary effects at transitions between geometric modes; and (5) a baseline depth trend reflecting the model's overall computational rhythm. The equation family is validated across eight decoder-only transformer models spanning five architecture families (GPT-2 Small/Medium/Large/XL, Llama 3 8B, Mistral 7B, OPT-6.7B, Pythia6.9B), parameter counts from 124M to 8B, and layer counts from 12 to 48. All eight models achieve R² = 0.83–0.97 with residual autocorrelation below 0.20. Three architecture-dependentcorrection classes emerge from residual diagnostics: (A) no correction needed, (B) localized spline knots for depth-specific dynamics, and (C) autoregressive closure for structured phase-specific dynamics unique to the sole parallel-block architecture. The paper also identifies an entropy-speed anti-correlation law (r = -0.84 to -0.87) linking routing weight uncertainty to representational velocity, and resolves a Simpson's paradox in the coupling force via geometry-conditional regression. An honest prospective failure — four of six pre-registered predictions confirmed for Pythia, leading to the discovery of the third correction class — is reported in full. This is the fifth and final paper in the GCI program: instrument (Paper 1) → emergence mechanism (Paper 2) → statistical mechanics (Paper 3) → normative principle via the Free Energy Principle (Paper 4) → dynamics (this paper). Paper 5 is self-contained and does not require the earlier papers, which remain unpublished.
No takes yet. Share an insight, caveat, or question.
Rod Robin Reyes Rodríguez (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: