Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowledge Graph Large Language Model (KG-LLM) for Link Prediction

Dong Shu, Tianle Chen, Mingyu Jin, Chong Zhang, Mengnan Du, Yongfeng Zhang

TL;DR

This work introduces KG-LLM, a framework that converts knowledge graphs into chain-of-thought natural language prompts and applies instruction fine-tuning to large language models for multi-hop link and relation prediction. By extracting multiple-hop paths, prompting with structured knowledge prompts, and leveraging in-context learning, KG-LLM aims to enhance reasoning, generalization, and ability to handle unseen prompts. The approach is validated on four real-world KG benchmarks, showing that KG-LLM outperforms traditional embedding and ablation baselines, with notable gains when using in-context learning for both link and relation tasks. The study highlights the potential of combining CoT reasoning, IFT, and ICL to improve generative KG reasoning, and outlines future work on reasoning transparency and prompt quality control.

Abstract

The task of multi-hop link prediction within knowledge graphs (KGs) stands as a challenge in the field of knowledge graph analysis, as it requires the model to reason through and understand all intermediate connections before making a prediction. In this paper, we introduce the Knowledge Graph Large Language Model (KG-LLM), a novel framework that leverages large language models (LLMs) for knowledge graph tasks. We first convert structured knowledge graph data into natural language and then use these natural language prompts to fine-tune LLMs to enhance multi-hop link prediction in KGs. By converting the KG to natural language prompts, our framework is designed to learn the latent representations of entities and their interrelations. To show the efficacy of the KG-LLM Framework, we fine-tune three leading LLMs within this framework, including Flan-T5, LLaMa2 and Gemma. Further, we explore the framework's potential to provide LLMs with zero-shot capabilities for handling previously unseen prompts. Experimental results show that KG-LLM significantly improves the models' generalization capabilities, leading to more accurate predictions in unfamiliar scenarios.

Knowledge Graph Large Language Model (KG-LLM) for Link Prediction

TL;DR

This work introduces KG-LLM, a framework that converts knowledge graphs into chain-of-thought natural language prompts and applies instruction fine-tuning to large language models for multi-hop link and relation prediction. By extracting multiple-hop paths, prompting with structured knowledge prompts, and leveraging in-context learning, KG-LLM aims to enhance reasoning, generalization, and ability to handle unseen prompts. The approach is validated on four real-world KG benchmarks, showing that KG-LLM outperforms traditional embedding and ablation baselines, with notable gains when using in-context learning for both link and relation tasks. The study highlights the potential of combining CoT reasoning, IFT, and ICL to improve generative KG reasoning, and outlines future work on reasoning transparency and prompt quality control.

Abstract

The task of multi-hop link prediction within knowledge graphs (KGs) stands as a challenge in the field of knowledge graph analysis, as it requires the model to reason through and understand all intermediate connections before making a prediction. In this paper, we introduce the Knowledge Graph Large Language Model (KG-LLM), a novel framework that leverages large language models (LLMs) for knowledge graph tasks. We first convert structured knowledge graph data into natural language and then use these natural language prompts to fine-tune LLMs to enhance multi-hop link prediction in KGs. By converting the KG to natural language prompts, our framework is designed to learn the latent representations of entities and their interrelations. To show the efficacy of the KG-LLM Framework, we fine-tune three leading LLMs within this framework, including Flan-T5, LLaMa2 and Gemma. Further, we explore the framework's potential to provide LLMs with zero-shot capabilities for handling previously unseen prompts. Experimental results show that KG-LLM significantly improves the models' generalization capabilities, leading to more accurate predictions in unfamiliar scenarios.
Paper Structure (32 sections, 1 equation, 5 figures, 3 tables)

This paper contains 32 sections, 1 equation, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Knowledge Graph Definition
  5. Multi-hop Link Prediction
  6. Multi-hop Relation Prediction
  7. Knowledge Prompt
  8. KG-LLM Framework
  9. Experiments
  10. Experimental Setup
  11. Datasets.
  12. Task splits.
  13. Comparing Baselines.
  14. Implementation Details.
  15. Metrics for Multi-hop Link Prediction.
  16. ...and 17 more sections

Figures (5)

  • Figure 1: A visual overview of KG-LLM framework
  • Figure 2: An Example of Prompt Used in the Multi-hop Link Prediction Training Process: Models processed through the ablation framework will be trained using the ablation knowledge prompt (left), whereas models processed via the KG-LLM framework will be trained on the KG-LLM knowledge prompt (right).
  • Figure 3: Overview of our knowledge prompts in the ablation and KG-LLM Frameworks: Ablation framework's knowledge prompts are in the first and third columns. KG-LLM framework's knowledge prompts are in the second and fourth columns.
  • Figure 4: Linear Relationship Between Complexity of Multi-Hop and Performance Score
  • Figure 5: Multi-hop Relation Prediction Performance Comparison: The left graph shows model performance under the ablation framework, while the right graph shows model performance under the KGLLM framework. Blue bars represent testing without ICL, and red bars represent testing with ICL.