Adaptive Retrieval and Evidence Calibration for Reliable Retrieval-Augmented Generation

Retrieval-augmented generation (RAG) can expose a response to incorrect evidence when a retrieved source is plausible but mismatched. This paper presents ACRE, a compact pipeline that couples query-dependent reranking depth, source-evidence confidence estimation, and a selective output decision. A BM25 candidate list is augmented with lexical similarity signals; logistic confidence models select one, three, or five candidates and determine whether a verbatim, source-attributed answer should be issued. We evaluate on 300 authentic news reports and 50 out-of-index reports from the Lee text corpora, using 1,400 reproducibly derived source-identification queries. On the 280-query held-out split, adaptive retrieval matches the BM25 top-1 source accuracy of 77.86% while requiring only 1.73 candidates per query in the assigned reranking depth, versus five for the fixed-depth alternative. With the development-selected abstention threshold, the system responds to 80.36% of test queries at 91.11% accepted-source accuracy; its expected calibration error is 0.039. The results demonstrate an economical approach to coupling retrieval efficiency with calibrated source attribution in an explicitly extractive RAG setting.