A Self-Controlled Benchmark of Retrieval-Augmented Generation for Large Language Models on Clinical Guideline Questions

This deposit provides the code, prompts, retrieval logs, and evaluation data for a self-controlled benchmark of retrieval-augmented generation (RAG) applied to clinical practice-guideline question answering. It includes a fully transparent retrieval pipeline (document parsing, structure-aware chunking, open multilingual embeddings, hybrid dense/lexical retrieval with cross-encoder reranking, and generation), the exact system and evaluation prompts, the chunked guideline text with provenance, per-query retrieval logs, the benchmark questions with gold-standard answers and guideline references, and the accompanying evaluation results (automated and human accuracy and hallucination scores). An analysis script reproduces the within-model retrieval effects. The benchmark is based on the German S3 guidelines AWMF 007-100OL (oral cavity carcinoma) and 007-091 (antiresorptive-associated jaw necrosis); the source guideline PDFs are not redistributed for copyright reasons and can be obtained from the AWMF using the register numbers provided. This deposit accompanies the associated journal article, which contains full methodological detail.