Improving Factual Grounding and Citation Reliability in Retrieval-Augmented Generation Systems

Retrieval-Augmented Generation (RAG) combines information retrieval with large language model (LLM) generation to answer questions using external documents. Although retrieved passages provide contextual evidence, their presence does not guarantee that every generated claim is supported or that a citation is correctly associated with the claim it accompanies. This paper presents the design and preliminary evaluation of a document-based RAG application integrating hybrid retrieval, claim-level evidence assessment, and citation-label validation. The application uses Angular, Java Spring Boot, PostgreSQL with pgvector, and locally hosted models through Ollama. Documents are extracted, divided into overlapping chunks, embedded using nomic-embed-text, and retrieved through a ranking function combining semantic and lexical relevance. The qwen2.5:3b-instruct model generates answers from retrieved context. A validation stage assigns evidence statuses and separately checks whether citation labels correspond to retrieved sources. Evaluation used 17 predefined questions over a five-page fictional employee handbook. Seven responses were classified as SUPPORTED and ten as NOT_SPECIFIED. Citation-label checking identified 12 VALID_LABELS responses and five MISSING citations. These results are preliminary: the experiment used one document and one recorded run, and citation-label consistency does not establish semantic entailment. The study treats evidence support and citation consistency as distinct reliability dimensions and identifies larger-scale comparative evaluation as future work.