Two Axes of Learned Representation: Where Structure Forms, and Whether It Persists Beyond the Data

We propose a two-axis framework for measuring what training adds to a neural network. Axis 1 (structure) asks where in the network learned, task-relevant structure is added; Axis 2 (generalization) asks whether that structure persists beyond the training data. We do not propose a reasoning metric; we propose two measurable axes we take to be necessary components of any future one. The two axes share a single unit that we operationalize as comparison — we model a representation by the task-relevant distinctions it makes linearly accessible, because that is directly measurable with probes, not because comparison is claimed to be the essence of representation. We provide three pieces of evidence. (1) Axis 1 on a real model: layer-wise logit/tuned-lens probing of a 1B Llama trained with partial backpropagation (BP on 6 of 24 layers) shows learned structure is localized — 80–99% concentrates in the BP window, while frozen random layers add exactly nothing (confirmed under tuned lens, ruling out coordinate-misalignment artifacts). Full BP, by contrast, spreads structure roughly uniformly. (2) Axis independence (calibration): in modular-arithmetic grokking, memorization (axis 1 saturates at ~225 steps) and generalization (axis 2 jumps thousands of steps later) are temporally separated in 6/6 runs, with four signals (test accuracy, OOD probe transfer, Fourier concentration, effective-rank drop) transitioning together at the grok point. (3) A post-hoc axis-2 instrument (distinct from the calibration in (2)): without any access to training dynamics, a probe-transfer procedure cleanly separates a generalizing model (score ≈0.99) from a memorizing one (≈0.35–0.41) and random init (chance), across two operations, three seeds, and two depths — and the score is graded, tracking partial generalization. A by-product: memorizing models that reach 100% behavioral train accuracy have representations that are linearly inconsistent even in-distribution, separating representation from behavior. This is a Stage-1 concept proposal with deliberately scoped evidence: measurements are pre-anchor (ordinal, not calibrated), the instrument demo is one synthetic task with an interpolation split, and the central LM result is at 1B scale. We state these limits explicitly and outline the path to a calibrated instrument.