Replication Instability in Experience Replay for Multi-Agent LLM Pipelines

Experience replay, the re-injection of highly rated past outputs as in-context exemplars, gives large language model (LLM) agents memory without parameter updates. In a multi-agent system such a store is the organization's memory, and retrieval a coordination mechanism. Its effect is typically measured in a single stochastic run, which this paper shows cannot support the claim drawn from it. In a five-agent LLM pipeline, after repairing three coordination failure modes that had prevented the loop from closing, fifteen controlled comparisons across two models produced no replay effect that reproduces. Significance did not survive replication even where the sign did. Single-run evaluations produced large, significant effects of opposite sign on the two models (Δ=-0.172, p = 0.041 and Δ=+0.147, p = 0.040), and neither survived replication. A pool-size comparison that looked decisive in single runs did not survive either. Ten independent runs of one identical configuration, five per model, spanned a range of 0.138 and disagreed about significance and, on one model, about the sign. What relevance-matched retrieval can return is bounded by the resolution of its key rather than by the size of the store, and we prove the bound for key-dominated scoring; the parameter the literature reports is not the one that governs the outcome. An objective task with a deterministic answer key reproduced the instability. We conclude with a seven-point protocol to make such evaluations informative rather than merely confident. Code, data, and the full call log are available at https://github.com/wjh55224545/replication-instability-in-experience-replay.