Answers are grounded in this paper and cite their sources.
Every RAG system must choose how many documents to retrieve per query. In practice this is an engineering guess — typically between 10 and 100 — with no theoretical justification. We prove this number is a mathematical constant determined by the embedding dimension: the optimal retrieval depth equals the square root of the dimension divided by its logarithm, which is asymptotically equal to the count of prime numbers up to the square root of the dimension.Below this threshold, each retrieved document contributes independent semantic information. Above it, retrieved documents become pairwise redundant — they are similar to the query and similar to each other — and marginal information drops to zero. This is the first rigorous geometric proof of the Maximal Marginal Relevance saturation point.The connection to prime numbers is not accidental. Both arise from the same mathematical mechanism: a Logarithmic Entropy Sieve, structurally identical to the Sieve of Eratosthenes. In Eratosthenes, prime divisors eliminate composites from integers. In our theory, spherical cap intersections eliminate redundant directions from high-dimensional embedding spaces. Both processes yield the same survivor density.Current RAG deployments using k between 20 and 100 waste 3 to 10 times the optimal compute for leading embedding models from OpenAI, Cohere, Google, and Meta. Validated across 5 datasets and 4 embedding dimensions.
No takes yet. Share an insight, caveat, or question.
Andres Sebaatian Pirolo (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: