Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast Think-on-Graph: Wider, Deeper and Faster Reasoning of Large Language Model on Knowledge Graph

Xujian Liang, Zhaoquan Gu

TL;DR

FastToG addresses hallucination and efficiency in graph-based retrieval augmented generation by enabling large language models to think 'community by community' over knowledge graphs. It introduces Local Community Search with modularity-based coarse pruning and LLMs-based fine pruning, plus two community-to-text methods (Triple2Text and Graph2Text) to convert graph structures into input for LLMs. Experiments on six real-world datasets show that FastToG yields higher accuracy and faster reasoning than prior GRAG methods, with improved explainability through structured community-level reasoning. The work highlights practical gains in scalable KG reasoning with LLMs, while noting Graph2Text can introduce hallucinations and suggesting directions for reducing such effects.

Abstract

Graph Retrieval Augmented Generation (GRAG) is a novel paradigm that takes the naive RAG system a step further by integrating graph information, such as knowledge graph (KGs), into large-scale language models (LLMs) to mitigate hallucination. However, existing GRAG still encounter limitations: 1) simple paradigms usually fail with the complex problems due to the narrow and shallow correlations capture from KGs 2) methods of strong coupling with KGs tend to be high computation cost and time consuming if the graph is dense. In this paper, we propose the Fast Think-on-Graph (FastToG), an innovative paradigm for enabling LLMs to think ``community by community" within KGs. To do this, FastToG employs community detection for deeper correlation capture and two stages community pruning - coarse and fine pruning for faster retrieval. Furthermore, we also develop two Community-to-Text methods to convert the graph structure of communities into textual form for better understanding by LLMs. Experimental results demonstrate the effectiveness of FastToG, showcasing higher accuracy, faster reasoning, and better explainability compared to the previous works.

Fast Think-on-Graph: Wider, Deeper and Faster Reasoning of Large Language Model on Knowledge Graph

TL;DR

FastToG addresses hallucination and efficiency in graph-based retrieval augmented generation by enabling large language models to think 'community by community' over knowledge graphs. It introduces Local Community Search with modularity-based coarse pruning and LLMs-based fine pruning, plus two community-to-text methods (Triple2Text and Graph2Text) to convert graph structures into input for LLMs. Experiments on six real-world datasets show that FastToG yields higher accuracy and faster reasoning than prior GRAG methods, with improved explainability through structured community-level reasoning. The work highlights practical gains in scalable KG reasoning with LLMs, while noting Graph2Text can introduce hallucinations and suggesting directions for reducing such effects.

Abstract

Graph Retrieval Augmented Generation (GRAG) is a novel paradigm that takes the naive RAG system a step further by integrating graph information, such as knowledge graph (KGs), into large-scale language models (LLMs) to mitigate hallucination. However, existing GRAG still encounter limitations: 1) simple paradigms usually fail with the complex problems due to the narrow and shallow correlations capture from KGs 2) methods of strong coupling with KGs tend to be high computation cost and time consuming if the graph is dense. In this paper, we propose the Fast Think-on-Graph (FastToG), an innovative paradigm for enabling LLMs to think ``community by community" within KGs. To do this, FastToG employs community detection for deeper correlation capture and two stages community pruning - coarse and fine pruning for faster retrieval. Furthermore, we also develop two Community-to-Text methods to convert the graph structure of communities into textual form for better understanding by LLMs. Experimental results demonstrate the effectiveness of FastToG, showcasing higher accuracy, faster reasoning, and better explainability compared to the previous works.
Paper Structure (37 sections, 6 equations, 9 figures, 8 tables, 2 algorithms)

This paper contains 37 sections, 6 equations, 9 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Algorithms of Community Detection
  4. Graph-based Retrieval Augmented Generation
  5. The Method
  6. Overview
  7. Local Community Search
  8. Community Detection on Subgraph
  9. Modularity-based Coarse-Pruning
  10. LLMs-based Fine-Pruning
  11. Reasoning
  12. Community-to-Text
  13. Experiments
  14. Experimental Setup
  15. Dataset and Evaluation Metric
  16. ...and 22 more sections

Figures (9)

  • Figure 1: Comparison of 1-w 1-d Graph RAG (a), n-w Graph RAG (b), n-d Graph RAG (c) and n-w n-d GraphRAG (d)
  • Figure 2: A general schema of the FastToG paradigm.
  • Figure 3: Average Depth versus Max size of community
  • Figure 4: Accuracy versus Max size of community
  • Figure 5: Accuracy (%) of two pruning methods
  • ...and 4 more figures