ReMindRAG: Low-Cost LLM-Guided Knowledge Graph Traversal for Efficient RAG

TL;DR

ReMindRAG presents a train-free memory replay mechanism integrated with LLM-guided KG traversal to balance retrieval effectiveness and cost in KG-RAG. By memorizing traversal experience within KG edge embeddings and combining node exploration with exploitation, it reduces LLM calls while preserving accuracy, and provides theoretical guarantees on memory capacity under semantic similarity. Experimental results across long-dependency and multi-hop QA tasks show consistent accuracy improvements (roughly 5–12 points) with substantial token-cost reductions (around 50%), plus robust self-correction and memory stability under various query conditions. The work highlights practical implications for scalable, memory-augmented RAG systems and points to future directions such as preloaded domain FAQs to accelerate initialization.

Abstract

Knowledge graphs (KGs), with their structured representation capabilities, offer promising avenue for enhancing Retrieval Augmented Generation (RAG) systems, leading to the development of KG-RAG systems. Nevertheless, existing methods often struggle to achieve effective synergy between system effectiveness and cost efficiency, leading to neither unsatisfying performance nor excessive LLM prompt tokens and inference time. To this end, this paper proposes REMINDRAG, which employs an LLM-guided graph traversal featuring node exploration, node exploitation, and, most notably, memory replay, to improve both system effectiveness and cost efficiency. Specifically, REMINDRAG memorizes traversal experience within KG edge embeddings, mirroring the way LLMs "memorize" world knowledge within their parameters, but in a train-free manner. We theoretically and experimentally confirm the effectiveness of REMINDRAG, demonstrating its superiority over existing baselines across various benchmark datasets and LLM backbones. Our code is available at https://github.com/kilgrims/ReMindRAG.

Figures (16)

• Figure 1: Overall Workflow. ReMindRAG constructs a KG from unstructured text. It memorizes LLM-guided traversal path, enabling fast retrieval when encountering similar queries subsequently.
• Figure 2: Illustration of LLM-Guided KG traversal in ReMindRAG. It iteratively expands the subgraph towards answer nodes through a process that combines node exploration and exploitation.
• Figure 3: Schematic Diagram of Memorizing.
• Figure 5: The case of memory replay under the Same Query setting. In the first search, no relevant info about 'Ed Wood' is found, but instead some unrelated details about 'Scott Derrickson' appear. After the search, the system enhances useful paths and penalizes irrelevant ones. The second search then focuses on the subgraph with the correct info about 'Scott Derrickson', achieving self-correction.
• Figure 6: The case of memory replay under the Different Query setting. In this case, a person's name is replaced in the second query, resulting in different answer subgraphs. However, the LLM is able to detect the incompleteness of the candidate answer subgraph and restart the traversal process. A complete demonstration of the entire process can be found in \ref{['appendix::complete-log']}.

Theorems & Definitions (4)

• Remark 1
• Remark 2
• Proposition 1
• Proof 1