Reliable Biomedical RAG with Answerability-Aware Hybrid Retrieval

Reliable retrieval-augmented generation (RAG) in biomedical settings must solve two coupled problems: the system must retrieve high-coverage evidence for supported questions, and it must abstain gracefully when the corpus does not contain sufficient evidence. We present an answerability-aware hybrid retrieval pipeline evaluated on the SciFact benchmark. The system first converts 5,183 scientific abstracts into 6,241 sentence-aware passages, then combines sparse BM25 retrieval with dense sentence-embedding retrieval using min-max normalized score fusion. In parallel, it constructs an answerability dataset from 1,109 SciFact queries augmented with out-of-domain and domain-similar hard negatives, and trains a BERT-based answerability gate to distinguish supported from unsupported questions. Empirically, hybrid retrieval yields the best evidence coverage, improving Recall@5 to 0.773 over 0.695 for BM25 and 0.749 for dense retrieval; Recall@10 rises to 0.826 over 0.748 and 0.800, respectively. The answerability classifier reaches 80.5% validation accuracy, with 97.5% recall on answerable queries and 64.3% recall on unsupported queries. We further formalize the offline training and online inference algorithms used by the system, making explicit how abstention, sparse candidate generation, dense rescoring, and retrieval-conditioned answering interact. Together, these results show that answerability-aware hybrid retrieval is a practical systems design for reliable biomedical RAG, especially when retrieval and abstention are evaluated as first-class components rather than incidental pre-processing steps.