Answers are grounded in this paper and cite their sources.
Experimental analysis reveals that network occupancy and structural spread govern emergence and forgetting in language models, indicating that training dynamics follow predictable kinetic laws.
When does a capability emerge during training? When is it forgotten? Both are open: emergence has been called unpredictable; forgetting gives no loss-curve warning. Both answers run through a quantity nobody measures: how much of the model is already in use. To measure it, reset one small piece of the network to its untrained state; if anything the model has learned gets worse, that piece was in use. The fraction in use is the occupancy, F. It is not loss in disguise: five models at the same loss have F from 0.006 to 1.00. With F in hand, emergence obeys a formula: a capability needing K parts, fed by data at frequency p, arrives at t* ≈ 1/(νS·p^K·X(F)^K), with X ≈ 1 below F ≈ 0.55, collapsing beyond 0.7. Until all K parts are right, the assembly yields nothing and sends back no gradient—no optimizer steers toward it. Arrival is a K-way coincidence, and coincidences have arithmetic. Frozen-in-advance constants forecast emergence on six held-out models to 5% median error. Forgetting follows different quantities. It ignores occupancy; what matters is how widely the capability has spread—measured by knocking out top components singly versus jointly—and how deep each copy sits: its logit margin, which sets the erosion rate. Concentrated in one head, a capability dies in 43 steps of starved training; spread into backups, it lasts 3,283; re-concentrated late, 96; across independent trainings, matched spread with deeper margin buys 10×. The spread law forecast before measurement—and hit. Why do neural networks obey laws that sound like crystallization? Because any system in which progress needs K things right at once, driven by fluctuation, on finite capacity, with structures that decay unless maintained, obeys them—as do crystals and, point by point, transformer training. One class; membership, not analogy. All results use open suites (14M–1B); these gauges are built for industrial scale.
No takes yet. Share an insight, caveat, or question.
Lei Dong (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: