Answers are grounded in this paper and cite their sources.
Controlled comparison evaluates retrieval quality and indexing costs in retrieval-augmented generation systems, suggesting distinct trade-offs in quality and cost.
This study investigates how chunk size and embedding model choice affect the retrieval quality, end-to-end answer quality, and indexing cost of Retrieval-Augmented Generation (RAG) systems. A controlled 3 × 3 experiment evaluates three chunk sizes (128, 256, and 512 words) with three open-source embedding models: BAAI/bge-small-en-v1.5, intfloat/e5-small-v2, and all-MiniLM-L6-v2. Experiments use an 800-question subset of the Stanford Question Answering Dataset (SQuAD 1.1). Retrieval performance is evaluated using Recall@5, while end-to-end question-answering performance is measured using token-level F1 and Exact Match (EM). Indexing cost is evaluated using indexing time, peak GPU memory (VRAM), and on-disk storage. Answer generation is performed using a locally served Llama 3.2 3B model on an NVIDIA GeForce GTX 1650 Ti with 4 GB VRAM. Results show that embedding model choice has a stronger effect on retrieval quality than chunk size. E5-small achieves the highest mean Recall@5 (0.938), followed by BGE-small (0.892) and MiniLM (0.880). The highest Recall@5 is obtained with 512-word chunks and E5-small (0.946). However, retrieval quality and end-to-end answer quality are only weakly correlated (r = +0.12), indicating that higher retrieval performance does not necessarily produce proportionally better generated answers. The highest F1 score is 0.415 for 512-word E5-small, while 256-word MiniLM provides the highest Exact Match rate (0.22). MiniLM also provides the lowest indexing cost. The findings demonstrate that chunk size and embedding model selection involve different quality-cost trade-offs in RAG systems. The results also highlight the importance of evaluating RAG pipelines using both retrieval-level and end-to-end generation metrics rather than relying on retrieval performance alone.
No takes yet. Share an insight, caveat, or question.
Ason Gautam (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: