Use Graph When It Needs: Efficiently and Adaptively Integrating Retrieval-Augmented Generation with Graphs

TL;DR

The paper tackles hallucination and outdated knowledge in large language models by combining retrieval-augmented generation (RAG) with graph-based retrieval (GraphRAG) in a dynamic, syntax-aware framework. It introduces EA-GraphRAG, which uses an offline heterogeneous graph, a lightweight MLP adapter, and a tri-routing policy to select between dense RAG, GraphRAG, or a fusion of both depending on query complexity. The approach demonstrates state-of-the-art QA performance across simple, complex, and mixed queries while reducing latency compared to pure GraphRAG methods, validating its practical value for heterogeneous workloads. These findings highlight the importance of adaptively routing queries to the retrieval paradigm best suited to the task, enabling robust, efficient knowledge-grounded QA in real-world settings.

Abstract

Large language models (LLMs) often struggle with knowledge-intensive tasks due to hallucinations and outdated parametric knowledge. While Retrieval-Augmented Generation (RAG) addresses this by integrating external corpora, its effectiveness is limited by fragmented information in unstructured domain documents. Graph-augmented RAG (GraphRAG) emerged to enhance contextual reasoning through structured knowledge graphs, yet paradoxically underperforms vanilla RAG in real-world scenarios, exhibiting significant accuracy drops and prohibitive latency despite gains on complex queries. We identify the rigid application of GraphRAG to all queries, regardless of complexity, as the root cause. To resolve this, we propose an efficient and adaptive GraphRAG framework called EA-GraphRAG that dynamically integrates RAG and GraphRAG paradigms through syntax-aware complexity analysis. Our approach introduces: (i) a syntactic feature constructor that parses each query and extracts a set of structural features; (ii) a lightweight complexity scorer that maps these features to a continuous complexity score; and (iii) a score-driven routing policy that selects dense RAG for low-score queries, invokes graph-based retrieval for high-score queries, and applies complexity-aware reciprocal rank fusion to handle borderline cases. Extensive experiments on a comprehensive benchmark, consisting of two single-hop and two multi-hop QA benchmarks, demonstrate that our EA-GraphRAG significantly improves accuracy, reduces latency, and achieves state-of-the-art performance in handling mixed scenarios involving both simple and complex queries.

Figures (3)

• Figure 1: An illustrative example showing that GraphRAG often underperforms vanilla RAG on simple single-hop queries, but excels on multi-hop queries requiring relational reasoning; conversely, vanilla RAG tends to perform better on single-hop queries while struggling with multi-hop reasoning.
• Figure 2: The overview of our EA-GraphRAG framework. In the (c) Decision Module stage, an MLP adapter produces a scalar complexity score $s(q)\in(0,1)$. Two thresholds route the query: $s(q)<\tau_L\Rightarrow$ Dense retrieval; $s(q)\ge\tau_H\Rightarrow$ Graph-based retrieval; $\tau_L\le s(q)<\tau_H\Rightarrow$ Fusion, where the documents retrieved by dense retrieval and graph-based retrieval are merged by weighted Reciprocal Rank Fusion (wRRF).
• Figure 3: Accuracy and efficiency comparison of adapters.