Simulation study demonstrates that content-addressed memory allocation trades destructive interference for retrieval difficulty in continual learning, suggesting an alternative to parameter...
A learner with bounded parameters and an unbounded stream must give something up: regularisation gives up plasticity, training on regardless gives up the past. A third option is to allocate, giving distinct regimes distinct storage, and the standing objection is that adding parameters per task defers the problem rather than solving it. We take that objection seriously by building the most committed version of allocation we could — storage addressed by content through a fixed random map, so no later learning can move where a fact lives; regimes inferred online with no task identifier, boundary or replay; and capacity appended rather than resized, so adding storage leaves existing weights and predictions bit-identical — and asking what it buys.Our reading is that allocation does not remove interference so much as convert it, trading destruction of old material for difficulty in retrieving among more of it. We argue this is the better failure, because retrieval difficulty is non-destructive and answerable by addressing better while interference is answerable only by spending parameters; we report the exchange rate we measured and the mitigations it implies. The same commitment has a price: an address fixed by content is precisely what keeps context out of retrieval.On a synthetic relational stream the mechanism retains substantially more than a monolithic control where regimes are distinguishable, allocates capacity in proportion toemitted content rather than vocabulary size, and holds per-token cost independent of context length. Other readings came out flat and we report them as open: six measurements in this study that read as results proved to have been produced by the measurement rather than the mechanism, and a synthetic source cannot exhibit what it was not built to contain.
No takes yet. Share an insight, caveat, or question.
Xinyu Yang (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: