ER-RAG: Enhance RAG with ER-Based Unified Modeling of Heterogeneous Data Source

Retrieval-Augmented Generation (RAG) enhances the reliability of large language models (LLMs) by integrating external evidence for question-answering tasks. However, when supporting evidence is scattered across heterogeneous data sources ( e.g., web pages, databases, and knowledge graphs), current RAG methods based on dedicated retrieval modules for each source face limitations like resource inefficiency in training modules, complex interaction among different sources, and lack of effective source selection. To address these challenges, we propose ER-RAG, a novel framework that allows efficient and effective evidence utilization. By observing that the evidence access patterns are restricted in RAG systems, ER-RAG introduces unified APIs based on the classic entity-relationship (ER) modeling to standardize the data access to heterogeneous data sources. ER-RAG then generates a chain of unified APIs to locate the desired evidence in a two-stage generation process, in which a direct preference optimization-tuned module first identifies optimal sources, and a dedicated module then generates API chains based on selected sources’ schema. Experiments show ER-RAG with an 8B LLM backbone performs comparably to commercial RAG pipelines using state-of-the-art LLMs. ER-RAG also outperforms other hybrid source competitors by 3.1% in LLM score and speeds up retrieval by 5.5X.