Less is More: Denoising Knowledge Graphs For Retrieval Augmented Generation

TL;DR

This work tackles the problem of noisy and redundant LLM-generated knowledge graphs in Graph-based RAG by proposing DEG-RAG, a denoising framework built on entity resolution and triple reflection. Through comprehensive experiments across four datasets and four Graph-based RAG baselines, it demonstrates that removing about $40\%$ of entities and relations and filtering unreliable triples yields consistent QA improvements and reduced KG size. Key insights show type-aware blocking, traditional KG embeddings, and direct merging often outperform alternative strategies, informing practical design choices for KG-based LLM applications. The results indicate that KG quality, rather than sheer size, drives retrieval and generation performance, offering a path toward more efficient and accurate KG-grounded LLM systems with broad applicability.

Abstract

Retrieval-Augmented Generation (RAG) systems enable large language models (LLMs) instant access to relevant information for the generative process, demonstrating their superior performance in addressing common LLM challenges such as hallucination, factual inaccuracy, and the knowledge cutoff. Graph-based RAG further extends this paradigm by incorporating knowledge graphs (KGs) to leverage rich, structured connections for more precise and inferential responses. A critical challenge, however, is that most Graph-based RAG systems rely on LLMs for automated KG construction, often yielding noisy KGs with redundant entities and unreliable relationships. This noise degrades retrieval and generation performance while also increasing computational cost. Crucially, current research does not comprehensively address the denoising problem for LLM-generated KGs. In this paper, we introduce DEnoised knowledge Graphs for Retrieval Augmented Generation (DEG-RAG), a framework that addresses these challenges through: (1) entity resolution, which eliminates redundant entities, and (2) triple reflection, which removes erroneous relations. Together, these techniques yield more compact, higher-quality KGs that significantly outperform their unprocessed counterparts. Beyond the methods, we conduct a systematic evaluation of entity resolution for LLM-generated KGs, examining different blocking strategies, embedding choices, similarity metrics, and entity merging techniques. To the best of our knowledge, this is the first comprehensive exploration of entity resolution in LLM-generated KGs. Our experiments demonstrate that this straightforward approach not only drastically reduces graph size but also consistently improves question answering performance across diverse popular Graph-based RAG variants.

Figures (6)

• Figure 1: Redundant concept synonyms for "LLMs" in a knowledge graph. Orange dashed lines indicate synonymic equivalences showing why these entities convey the same meaning as "LLMs".
• Figure 2: The overall framework of entity resolution for knowledge graphs ER_survey.
• Figure 3: Impact of different entity resolution components on Graph-based RAG performance.
• Figure 4: Influence of entity reduction ratio on Graph-based RAG performance.
• Figure 5: Ablation study on the performance of the full denoising method against versions without entity resolution, without triple reflection, and with random entity merging.