Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CogMG: Collaborative Augmentation Between Large Language Model and Knowledge Graph

Tong Zhou, Yubo Chen, Kang Liu, Jun Zhao

TL;DR

This work introduces a collaborative augmentation framework, CogMG, leveraging knowledge graphs to address the limitations of LLMs in QA scenarios, explicitly targeting the problems of incomplete knowledge coverage and knowledge update misalignment.

Abstract

Large language models have become integral to question-answering applications despite their propensity for generating hallucinations and factually inaccurate content. Querying knowledge graphs to reduce hallucinations in LLM meets the challenge of incomplete knowledge coverage in knowledge graphs. On the other hand, updating knowledge graphs by information extraction and knowledge graph completion faces the knowledge update misalignment issue. In this work, we introduce a collaborative augmentation framework, CogMG, leveraging knowledge graphs to address the limitations of LLMs in QA scenarios, explicitly targeting the problems of incomplete knowledge coverage and knowledge update misalignment. The LLMs identify and decompose required knowledge triples that are not present in the KG, enriching them and aligning updates with real-world demands. We demonstrate the efficacy of this approach through a supervised fine-tuned LLM within an agent framework, showing significant improvements in reducing hallucinations and enhancing factual accuracy in QA responses. Our code and video are publicly available.

CogMG: Collaborative Augmentation Between Large Language Model and Knowledge Graph

TL;DR

This work introduces a collaborative augmentation framework, CogMG, leveraging knowledge graphs to address the limitations of LLMs in QA scenarios, explicitly targeting the problems of incomplete knowledge coverage and knowledge update misalignment.

Abstract

Large language models have become integral to question-answering applications despite their propensity for generating hallucinations and factually inaccurate content. Querying knowledge graphs to reduce hallucinations in LLM meets the challenge of incomplete knowledge coverage in knowledge graphs. On the other hand, updating knowledge graphs by information extraction and knowledge graph completion faces the knowledge update misalignment issue. In this work, we introduce a collaborative augmentation framework, CogMG, leveraging knowledge graphs to address the limitations of LLMs in QA scenarios, explicitly targeting the problems of incomplete knowledge coverage and knowledge update misalignment. The LLMs identify and decompose required knowledge triples that are not present in the KG, enriching them and aligning updates with real-world demands. We demonstrate the efficacy of this approach through a supervised fine-tuned LLM within an agent framework, showing significant improvements in reducing hallucinations and enhancing factual accuracy in QA responses. Our code and video are publicly available.
Paper Structure (15 sections, 3 figures, 1 table)

This paper contains 15 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Framework Design
  3. Querying Knowledge Graph
  4. Processing Result
  5. Knowledge Graph Evolution
  6. Implementation and Usecase
  7. Model and Components
  8. System and Use Case
  9. Experiments
  10. Related Work
  11. Knowledge Base Question Answering
  12. Updating Knowledge Graph
  13. Conclustion
  14. Limitations
  15. Acknowledgments

Figures (3)

  • Figure 1: Left part: A schematic diagram illustrating the overall design of the collaborative augmentation framework CogMG, involving LLM and KG. Right part: We implement CogMG using an agent-based framework, with each module designed to be plug-and-play to ensure generalizability.
  • Figure 2: A complete example of how the system handles queries not found in the knowledge graph and the processing of related knowledge.
  • Figure 3: System screenshot.