HierCache-RAG: Fast Retrieval-Level Caching for GraphRAG

Graph Retrieval-Augmented Generation (GraphRAG) improves multi-hop reasoning in Large Language Models(LLMs) by grounding generation in knowledge-graph structure, but most systems re-traverse the graph on everyquery, wasting computation whenever queries repeat or overlap semantically. We propose HierCache-RAG, ahierarchical retrieval-level caching framework that reuses graph-derived relational context through three cascadinglayers – an Exact Query Cache, a Semantic Query Cache, and a One-Hop Graph Cache – before falling back tograph traversal. Unlike prior work that caches at the generation or key-value stage, HierCache-RAG caches directlyat the retrieval stage. On DBLP, it reaches 2.03 ms effective retrieval latency (2.32/10.70 faster thanNetworkX/Neo4j) with competitive answer quality; on the more semantically diverse arXiv dataset, it reaches1.29 ms (up to 61.43 faster) while also achieving the highest answer quality of any system tested. These gains holdconsistently from 30K to 70K papers, showing that retrieval-level caching makes interactive GraphRAG practicalwithout trading away accuracy.