A Trained Fast-Weight Memory: Continual Rule Binding at Inference Without Backward

Continual learning at inference usually means test-time training (TTT): gradient steps on a clone of the model. We study the alternative the fast-weight literature has long promised: a small bank of vectors, written by the forward pass itself, that modulates subsequent computation — no backward, no optimizer, no weight copy. On a keyed multi-turn rule task (a fresh modular rule per conversation, unseen queries, K=2 concurrent rules), a 3M-parameter transformer with an 8-slot bank learns to (i) install a never-trained rule from a single 13-token presentation (held-rule accuracy 0.79–1.00 across two seeds; chance 0.008), (ii) retain it across turns and slot eviction, and (iii) replace it mid-conversation in one forward pass (post-switch accuracy 0.95 train / 0.78 held; old-rule persistence 0.000). Head-to-head on the same conversations, TTT with a full learning-rate sweep memorizes its adaptation examples (pair accuracy 0.99) yet transfers exactly nothing to unseen queries, while costing 138× more per rule update and destroying 62% of the untouched concurrent rule; in-window ICL is also at chance — the bank is the model's only functional adaptation pathway. Crucially, none of this is emergent from the architecture: the same architecture trained without mid-conversation rule switches perseverates completely (old-rule persistence 1.000 zero-shot). Memory policy — what to keep, when to overwrite, how to write on a dirty bank — is a trained behaviour, installed by randomizing conversation structure at training time. We map the boundary (an untrained rule family defeats bank and TTT equally: the limit is the meta-learned envelope, not the mechanism) and report a seed-level bifurcation between selective-update and flush-and-rewrite replacement policies. Training the read/write circuit requires breaking an ignore-the-bank fixed point; we give the recipe (teacher-forced code bootstrap with annealed blending, mastery-gated rule curriculum) and the diversity threshold below which the read memorizes instead of generalizing. Version of record of the repository master at tag v0.2-preprint