Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Relations Prediction for Knowledge Graph Completion using Large Language Models

Sakher Khalil Alqaaidi, Krzysztof Kochut

TL;DR

This work tackles relation prediction for knowledge graph completion by fine-tuning a large language model (Llama 2) on a multi-label sequence classification task that uses only node names as input, enabling inductive generalization to unseen nodes. The proposed RPLLM approach demonstrates competitive or superior performance on FreeBase and WordNet benchmarks, with detailed analysis of inductive settings and failure modes. By relying solely on textual node identifiers, the method highlights the potential of LLM-based signals for RP in open-world KG completion while maintaining a streamlined input pipeline. The results suggest that minimal textual cues processed by LLMs can effectively infer plausible relations, offering a scalable direction for RP in large-scale KGs.

Abstract

Knowledge Graphs have been widely used to represent facts in a structured format. Due to their large scale applications, knowledge graphs suffer from being incomplete. The relation prediction task obtains knowledge graph completion by assigning one or more possible relations to each pair of nodes. In this work, we make use of the knowledge graph node names to fine-tune a large language model for the relation prediction task. By utilizing the node names only we enable our model to operate sufficiently in the inductive settings. Our experiments show that we accomplish new scores on a widely used knowledge graph benchmark.

Relations Prediction for Knowledge Graph Completion using Large Language Models

TL;DR

This work tackles relation prediction for knowledge graph completion by fine-tuning a large language model (Llama 2) on a multi-label sequence classification task that uses only node names as input, enabling inductive generalization to unseen nodes. The proposed RPLLM approach demonstrates competitive or superior performance on FreeBase and WordNet benchmarks, with detailed analysis of inductive settings and failure modes. By relying solely on textual node identifiers, the method highlights the potential of LLM-based signals for RP in open-world KG completion while maintaining a streamlined input pipeline. The results suggest that minimal textual cues processed by LLMs can effectively infer plausible relations, offering a scalable direction for RP in large-scale KGs.

Abstract

Knowledge Graphs have been widely used to represent facts in a structured format. Due to their large scale applications, knowledge graphs suffer from being incomplete. The relation prediction task obtains knowledge graph completion by assigning one or more possible relations to each pair of nodes. In this work, we make use of the knowledge graph node names to fine-tune a large language model for the relation prediction task. By utilizing the node names only we enable our model to operate sufficiently in the inductive settings. Our experiments show that we accomplish new scores on a widely used knowledge graph benchmark.
Paper Structure (17 sections, 5 equations, 1 figure, 6 tables)

This paper contains 17 sections, 5 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Knowledge Graph Completion
  4. Language Models
  5. Methodology
  6. Text Encoding
  7. Llama 2
  8. Experiments
  9. Experimental Setup
  10. Evaluation Metrics
  11. Comparison Models
  12. Main Results
  13. Inductive Settings
  14. Failure Analysis
  15. Conclusion
  16. ...and 2 more sections

Figures (1)

  • Figure 1: Our model