Who Knew What, When, and Why: A Replayable, Bitemporal Memory Substrate for AI Agents

AI agents now produce work faster than the people responsible for that work can track it. The memory systems attached to those agents have become the de facto record of decisions, yet none of the deployed architectures can answer the questions any organization eventually asks of a record: who learned this fact, when, from what source, and what belief did it replace. We analyze twelve memory systems across the current design space and find a shared root cause: they treat memory as a mutable cache of extracted facts, so provenance is destroyed at write time and history is destroyed at update time. We propose the Bitemporal Provenance Memory Architecture (BPMA). Its core commitment is a canonical, append-only event record from which every recall structure, working memory, semantic facts, summaries, graphs, and retrieval indexes, is a derived, versioned, replayable projection. Every Memory Event and every extracted Claim carries its principal (human, agent, or pipeline, with model version), bitemporal coordinates (event time and ingestion time), and lineage to its sources. We release Memora, an open-source reference implementation, and AuditEval, a benchmark of five provenance query classes that existing conversational benchmarks do not test. On AuditEval, Memora answers 100% of origin, supersession, attribution, belief-at-time, and contamination queries by construction, while an otherwise identical destructive-update store answers 19.2%, 19.2%, 100%, 38.1%, and 0%. Single-threaded SQLite ingestion sustains about 5,800 events per second with sub-millisecond current-belief lookups at 50,000 events, and full projection rebuilds are bit-identical across replays. Code and benchmark: https://github.com/Nifty0x/memora