PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
July 13, 20260 citationsOpen Access

A random-matrix account of structure formation in neural networks

View Full Paper
JBJ. Arturo Ornelas Brand

Key Points

  • This research aims to quantify the relationship between random-matrix theory and structure formation in neural networks.
  • Examined the impact of architectural decoupling on level repulsion in neural networks.
  • Analyzed two pathways to the second Dyson class through neural network tasks and structural routes.
  • Measured eigenvector transitions indicating multifractal characteristics across different neural network configurations.
  • Isolated architectural decoupling as a causal variable for level repulsion in neural networks.
  • Demonstrated the robustness of the multifractal dimension D2 = 0.76 across five network configurations.
  • Identified a strong correlation (0.97) between eigenvalue and eigenvector statistics during training.

Abstract

A vibrating plate and a training neural network admit the same description: a control parameter drags the system's spectrum between universal fixed points of random matrices. We make this account quantitative in neural networks across the random-matrix signatures the plate program identifies. (i) Level repulsion (Poisson-to-GOE) does not transfer to dense networks — a clean double-negative in both weights and the exact Hessian — but is fabricable in architectures with weakly-coupled symmetry sectors forced to cooperate; we isolate the causal variable (architectural decoupling), give a continuous dose-response, and a capacity-attenuation law that explains why large dense models do not exhibit it. (ii) The second Dyson class (GOE-to-GUE, time-reversal breaking) is reachable two ways: the causal mask of a transformer is an exact, scale-invariant structural route (GPT is robustly unitary; BERT is orthogonal because its task is symmetric), and a directional task is a genuine learned route that, given positional capacity, reaches the same non-reciprocity. We report a negative: the large-width non-reciprocity of a bidirectional model is partly generic to capacity. (iii) At the eigenvector level — the discriminant the plate program says decides the localized/extended question ("Gap A") — the network's fabricated transition is genuinely Rosenzweig-Porter: multifractal with fractal dimension D2 = 0.76 in (0,1), not level-spacing mimicry. The exponent is robust across architectures: re-measured over five configurations, D2 stays inside a Rosenzweig-Porter band of 0.56–0.80 — never localized, never ergodic — so the qualitative claim generalizes while 0.76 is a representative value. And the two signatures are one transition: tracked on the same exact Hessian over a single training run, the eigenvalue statistic (the spacing ratio) and the eigenvector statistic (D2) co-transition (correlation 0.97, midpoints within 4% of training) — the dynamical signature of genuine Rosenzweig-Porter rather than spacing mimicry. The bridge is real, specific, and bounded by structure. Tier-2: a shared mathematics, not a physical identity.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

J. Arturo Ornelas Brand (2026) studied this question.

synapsesocial.com/papers/6a548219475c38bf615a5a94https://doi.org/10.5281/zenodo.21314468
Ask AI
Helpful
Bookmark
Share
View Full Paper