Understand Once, Retrieve Many Times: SAADI for Single-Pass Retrieval of Distributed Evidence in Long Documents

Long-document retrieval is challenging when relevant evidence is distributed across distant passages or requires multiple reasoning hops, especially under a limited retrieval budget. Single-pass top-k retrievers offer predictable query-time computation and latency, whereas agentic systems seek higher recall by reading documents over multiple steps. We shift progressive document understanding to index time, enabling single-pass top-k retrieval without query-time large language model (LLM) calls or adaptive trajectories. SAADI (Stateful Accumulation of Document Information) constructs a query-agnostic document state at index time using one state-update call per chunk. Each call extracts entities from the current chunk, conditioned on records recalled from entity memory and a hierarchical summary of the preceding content. Each chunk is linked to its resulting entity records and covering summary, while the original chunks remain the evidence and retrieval units. At query time, two auxiliary scores derived from these links are fused with the score of a sparse or dense base retriever in a single top-k pass. SAADI's index-time LLM calls, including summary updates, are fixed before queries arrive, requiring at most 1.25 calls per chunk in the default configuration. Each call processes context that grows only logarithmically with document length, and new chunks can be appended without rebuilding the existing state. Across four datasets and five retrievers, SAADI improves HR@10 over its base retrievers by 0.3 to 12.4 percentage points (1.1 to 4.0 points with BM25). It ranks first or second in 34 of 40 dataset–retriever–depth combinations and outperforms agentic retrievers in 16 of 20 comparisons at comparable average evidence exposure. These gains persist for contexts up to 256K tokens and across all evaluated evidence-hop counts. Unlike agentic retrieval, whose costs recur with every query, SAADI incurs a one-time indexing cost. Beyond a reported break-even number of queries per document, measured in both model calls and tokens, cumulative agentic costs exceed SAADI's construction cost. Ablation studies confirm that entity memory and hierarchical summaries independently improve retrieval, with their combination consistently achieving the best results. These findings demonstrate that document-level connections supporting distributed and multi-hop evidence retrieval can be established once at index time, with a predictable cost before the first query. Code and data: https://github.com/ZeinabTaghavi/SAADI