Does a low Fourier-restricted loss reveal early task-aligned structure, or reflect the construction of the measuring operator? We examine thirty modular-addition transformers using all 12,030 saved states. Frequencies selected from the current training logits, without terminal-state information, reach a common 0.1-nat test threshold before the original predictions in 30/30 models when coefficients are estimated across disjoint test folds. The median lead is 1,050 epochs, with a pointwise 95% median interval of 900–1,200. Cross-fitted scalar temperature calibration leads in only 2/30 models, with median zero. At the Fourier crossings, original answer accuracy is 27.6–55.5%; a classification-error bound excludes any nonnegative scalar rescaling as an explanation of the earlier persistent crossings on the saved grid in all thirty models. Energy-ranked non-target and wrong-direction controls do not cross the primary threshold. A secondary selector frozen at epoch 1,000 leads in 28/30, illustrating a limitation of fixing frequencies early. Separately, complete-basis fitting can reconstruct low-loss test predictions from a memorization-only logit table with training margin 40, while the original test predictions remain uniform. The model advantage survives a control that rejects this construction. These exploratory results support early, transferable structure in held-out outputs under a specified task-informed decoder; they do not date the causal formation of an internal circuit. Working preprint, not peer reviewed. This deposit contains the author-named manuscript only. Raw checkpoints, training code and the private derived-output review archive are not included. No journal or conference acceptance is claimed.
No takes yet. Share an insight, caveat, or question.
Ali Noori (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: