Evaluating the Impact of Cross-Encoder Re-Ranking on Retrieval-Augmented Generation Performance

Retrieval-Augmented Generation (RAG) systems ground language model outputs in retrieved evidence, but dense retrieval alone does not guarantee that the most answer-relevant passages reach the generator. This paper presents a controlled evaluation of cross-encoder re-ranking in a dense-retrieval RAG pipeline. We implement two systems that are identical except for the presence of a BERT-based cross-encoder re-ranker inserted between the retriever and generator. Evaluating on 200 Natural Questions validation examples over a corpus of 75,000 Wikipedia passages, we find that re-ranking improves Exact Match from 0.075 to 0.095 (+26.7%) and Token F1 from 0.029 to 0.040 (+39.3%). Absolute scores reflect the constrained experimental setting (a 75,000-passage corpus and a 3B-parameter generator), and the relative improvements are the primary contribution. The latency cost is approximately 4.3 seconds per query on CPU hardware, reducible to under 200ms with GPU deployment. To our knowledge, no prior work has isolated the causal effect of inserting a frozen, off-the-shelf cross-encoder re-ranker into an otherwise unchanged RAG pipeline, keeping retriever, generator, prompt, and corpus fixed, while reporting both retrieval and generation outcomes in the same experiment. Our results suggest re-ranking provides a meaningful and consistent improvement to RAG answer quality.