CommunityKG-RAG: Leveraging Community Structures in Knowledge Graphs for Advanced Retrieval-Augmented Generation in Fact-Checking

TL;DR

This work tackles misinformation by integrating Knowledge Graphs with Retrieval-Augmented Generation through a zero-shot framework, CommunityKG-RAG, which leverages KG community structures and multi-hop paths for evidence retrieval in fact-checking. It builds a KG from fact-checking articles, detects communities with the Louvain method, and uses node- and sentence-embedding strategies (BERT/Sentence-BERT) to rank and select relevant communities and sentences for RAG. The approach yields significant accuracy gains on the MOCHEG dataset, with a peak of 56.24% accuracy using a 25% top-community and 100% sentence-selection setting, outperforming No Retrieval, Semantic Retrieval, and KAPING baselines without fine-tuning. While showing strong zero-shot benefits, the method incurs higher computational costs and depends on high-quality entity recognition, suggesting avenues for efficiency improvements and domain-specific adaptations.

Abstract

Despite advancements in Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems, their effectiveness is often hindered by a lack of integration with entity relationships and community structures, limiting their ability to provide contextually rich and accurate information retrieval for fact-checking. We introduce CommunityKG-RAG (Community Knowledge Graph-Retrieval Augmented Generation), a novel zero-shot framework that integrates community structures within Knowledge Graphs (KGs) with RAG systems to enhance the fact-checking process. Capable of adapting to new domains and queries without additional training, CommunityKG-RAG utilizes the multi-hop nature of community structures within KGs to significantly improve the accuracy and relevance of information retrieval. Our experimental results demonstrate that CommunityKG-RAG outperforms traditional methods, representing a significant advancement in fact-checking by offering a robust, scalable, and efficient solution.

Figures (2)

• Figure 1: Comparison between no retrieval, semantic retrieval, and CommunityKG-RAG. The no retrieval and semantic retrieval fail to provide sufficient context, while our proposed method, CommunityKG-RAG, is able to by leveraging multi-hop knowledge graph information in the retrieval process enhancing accuracy and relevance.
• Figure 2: Workflow of CommunityKG-RAG