One Knob for Two Bottlenecks: Decay-Modulated Fast-Weight Memory Unifies Long-Context and Continual Learning

Two problems dominate the scaling of sequence models: the O(N²) cost of long-context attention, and catastrophic forgetting when models must learn online. We argue these are the same problem — both concern how a model stores its past — and that a single mechanism addresses both: a fixed-size, decaying, locally-written associative memory M read out as y = Mq. Treating backpropagation as a target rather than a physical process, we study a three-factor (Hebbian × neuromodulator) update with decay, Ṁ = −αM + η δ v k⊤, which subsumes linear attention, fast-weight programmers, and gated state-space models. We show (i) the decay rate α is a single knob that simultaneously sets the effective context length and the stability/plasticity balance; (ii) a closed-form edge-of-stability condition under which the neuromodulator δ provably prevents both explosion and vanishing, with no gradient clipping; (iii) the error-correcting (delta) form of the rule is online Widrow–Hoff and converges to the target associative map; and (iv) freezing the slow weights confines catastrophic forgetting to the bounded memory. We validate all four claims with small, reproducible experiments, and report two honest negative results: parameter-anchoring (EWC) fails to relax the frozen-weight assumption where rehearsal succeeds, and a nonlinear read-out preserves stability but forfeits the exact least-squares guarantee. Part I of the series "Toward Governed Adaptive Memory." Project: https://grafomem.com