The Value of Perfect Information about the Query in Budget-Constrained LLM Memory

Under token budgets, memory systems for Large Language Models (LLMs) select historical information before receiving a query (query-agnostic selection) or after receiving it (query-aware selection). To isolate the value of knowing the query, we idealize the continuous text of each history as a sequence of fixed chunks and take the chunks each query needs, its gold evidence, as known. A selection covers a query if it contains all of that query's gold evidence. We measure the Expected Value of Perfect Information (EVPI) about the query as the difference in optimal expected coverage between the two selections. EVPI is zero exactly when all queries that fit the budget alone also fit it together. On LoCoMo, FairytaleQA, and ContractNLI, peak EVPI reaches 51.27–87.67 percentage points at budgets of approximately 1%–17% of median history length. At these budgets, two LLM readers are 40.74–55.25 percentage points more accurate with optimal query-aware than with optimal query-agnostic selection on the evaluation sets. The advantage on LoCoMo and FairytaleQA persists after sampled human review. Under automatic scoring, the accuracy difference across budgets approximately follows the coverage difference scaled by each reader's gain from gold evidence. Readers also answer from partial evidence, which coverage leaves uncredited, so coverage and accuracy can rank selection policies oppositely. On ContractNLI at 160 tokens, dense retrieval covers fewer queries than optimal query-agnostic selection yet gives one reader 13.51 percentage points higher accuracy, mostly through partial evidence. Partial evidence adds few wrong answers on ContractNLI and many on LoCoMo. This record contains the preprint.