Answers are grounded in this paper and cite their sources.
A looped transformer applies one block of layers many times. It is trained either at a fixed number of loops r or at a number of loops drawn at random for every sequence. We measure the dynamics inside the loop of 51 looped language models trained on the same data, and find that a loss placed at one fixed depth locks the dynamics to that depth. Before r, the loop takes Euler-like steps: each step shrinks as r grows, and loops of depth 6 to 24 trace the same step profile in r steps. At r, the loop brakes: its Jacobian contracts most within one loop of r in 28 of the 31 models trained at a fixed depth through every loop, including all 27 with r ≤ 16. Past r, the step keeps rotating in a plane. At r = 6 to 12 the state swings away and back and the loss rises and falls; one turn takes about two training depths. Scaling every update by r/r′ moves this landing to loop r′ = 1.5r or 2r without retraining, and keeps the downstream accuracy of loop r. In our controls, none of these signatures appears when the depth is random, even with backpropagation through every loop, or when the loss is placed at every loop. Between r and 2r the loss of all 12 such models rises by at most 0.003 nats, against 0.02 to 0.51 nats in all 31 fixed-depth models trained through every loop. The recipes also differ in contraction, which may mediate the effect. At matched contraction, the loss of a damped fixed-depth loop still rises by 0.19 to 0.23 nats from r to 2r, that of a damped random-depth loop by at most 0.015. We prove that a loss averaged over depths confines the state in the directions the readout uses. Every registered test had its threshold written before it was run; the few that followed a first look at the data are flagged.
No takes yet. Share an insight, caveat, or question.
Leonardo Cofone (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: