Answers are grounded in this paper and cite their sources.
Demonstrates improved prediction accuracy in long-context question answering, suggesting a new method for efficient information retrieval.
Large Language Model (LLM) question answering (QA) is essential for various real-world applications. As the demand for LLMs in domain-specific, up-to-date, and private scenarios increases, equipping them with large external corpus for QA, referred to long-context LLM QA, becomes critical. However, existing methods struggle to balance the comprehensive retrieval of scattered information required for accurate QA with the high token costs of long-context processing. We identify two major causes of this bottleneck: (1) limited retrieval coverage under complex queries and large input scales, and (2) repeated document access enforced by rigid query-by-query processing. In this paper, we introduce a cost-effective Retrieval-Augmented Generation Sketch (RAG-Sketch) for long-context LLM QA. To enable more thorough information retrieval without exhaustive context traversal, we propose a new skyline retrieval module incorporating query decomposition into a pre-defined schema. This approach retrieves Pareto optimal chunks by jointly considering semantic similarity and the presence of key information within the decomposed queries. Based on retrieved chunks, we formally define a set cover chunk merging optimization problem and proposes a greedy algorithm with theoretical guarantees that effectively reduces redundant access. Our experiments on widely used long-context QA benchmarks demonstrate that RAG-Sketch improves prediction accuracy by 24.5% while reducing token costs by 58.5% compared with the state of the art.
No takes yet. Share an insight, caveat, or question.
Zhu et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: