Two recent results, one from statistical physics and one from machine learning, look completelyunrelated at rst glance. The rst reports that pushing a magnet through its phase transitiontakes more energy than it should and the extra energy follows a curious mathematical law.The second observes that very large neural networks behave strangely near a critical thresholdwhere the number of parameters equals the number of data points. The accompanying paperargues that these two phenomena are, secretly, the same mathematical event: a system beingdragged through a tipping point faster than it can keep up. This summary explains what thatmeans, why it matters, and what new things we can now predictincluding a precise predictionabout the timing of grokking in articial neural networks.
Alfredo Sepulveda-Jimenez (Sat,) studied this question.