A U-shape in dot-product attention under gradient flow

We prove that gradient flow on a single dot-product self-attention layer with tied unembedding, trained on cross-entropy loss in a two-token setting, exhibits a non-monotone trajectory in the Frobenius norm of its interaction matrix: starting from any sharp-wrong initialization in which both tokens initially attend strongly to the wrong target, there exists T₁ ∈ (0, ∞) such that ||M(t)||F² is strictly decreasing on [0, T₁] and diverges to +∞ as t → ∞. The proof rests on a row-sum conservation law for the logit matrix L = X M XT, which pins d² - 2 scalar invariants of M at initialization and reduces the effective dynamics to two scalar coefficients. Borrowing the blacksmith's vocabulary, training proceeds in three phases: heating (parameter contraction on [0, T₁]), forging (unbounded growth for t > T₁), and cooling (the asymptotic regime established in prior work). We characterize the first two phases.