PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
July 14, 20260 citationsOpen Access

Experience to Competence Without Weight Updates: Pre-Registered Falsifiable Gates for Agent Memory

View Full Paper
MRMatthias Raviotta

Key Points

  • Evaluate the effectiveness of various memory mechanisms in AI agents without updating weights.
  • Implemented five pre-registered falsifiable gates for memory assessment.
  • Used local consumer hardware with two memory mechanisms: selective consolidation and frozen agent.
  • Analyzed performance across 40 unseen episodes and compared with a memoryless control and oracle.
  • Selective consolidation compresses exposures from 350 to 150 memories with statistical parity.
  • First-try success increases from 0.45 to 1.00 in the agent using memory; control remains at 0.00 and oracle achieves 0.95.

Abstract

We subject a biologically inspired memory system for LLM agents (episodic traces, repetition strengthening, consolidation) to five pre-registered falsifiable gates with floor/ceiling arms and bootstrap CIs, all on consumer hardware with local models. Two popular mechanisms fail: repetition-weighted ranking loses to vanilla cosine retrieval (−0.44/−0.18, cue-specificity violation), and cosine similarity cannot separate same-fact paraphrases from same-template distractors (overlapping distributions), making embedding-only deduplication structurally unsound. Two mechanisms survive: selective consolidation compresses 350 exposures into 150 memories at statistical parity with store-everything; and a frozen 9B agent coupled to the memory converts API error messages into transferable competence, first-try success rises from 0.45 to 1.00 across 40 consecutive unseen episodes, while a memoryless control stays at 0.00 and an oracle reaches 0.95. Weights are never updated: memory is the only learning channel. Transfer replicates 3/3 seeds. We argue agent memory research needs control arms and pre-registration more than new architectures.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Matthias Raviotta (2026) studied this question.

synapsesocial.com/papers/6a55d11a5aafca87247f82b4https://doi.org/10.5281/zenodo.21323672
Ask AI
Helpful
Bookmark
Share
View Full Paper