Comparative benchmark reveals agentic memory maintains long-term conversational recall on consumer hardware, suggesting viable private local deployment without API models.
Key Points
To determine whether small local language models running on a single consumer GPU can manage long-term conversational memory without relying on API-hosted models.
Evaluated four memory strategies (no memory, full-history stuffing, raw-turn retrieval, and an agentic protocol extracting typed, dated, deduplicated facts) across 1,986 questions from 10 multi-session conversations in the LoCoMo benchmark.
Varying local extractor model size from 3B to 14B parameters on a 12 GB GPU, scoring outputs using an independent LLM judge and performing a hand-verified audit on 200 stored memories.
Full-context stuffing accuracy collapsed by 76% once context limits were exceeded, whereas agentic memory maintained consistent recall and outperformed stuffing 2.0x on the 39% of out-of-window questions while using ~67x less context.
Temporal grounding scaled with parameter size, establishing a minimum viable local model size of 7B to 14B parameters for reliable memory extraction, with human evaluation finding 199 of 200 stored facts faithful to source turns and zero fabrications.
Cite This Study
Muhammad Abdullah Shaheer (2026) studied this question.