Exploring Large Language Models for Knowledge Graph Completion
Liang Yao, Jiazhen Peng, Chengsheng Mao, Yuan Luo
TL;DR
This work introduces KG-LLM, a framework that treats knowledge graph triples as textual sequences and leverages instruction tuning of open large language models (e.g., LLaMA, ChatGLM) to perform KG completion tasks. By formulating triple classification, relation prediction, and entity prediction as prompts, KG-LLM achieves state-of-the-art results on standard benchmarks and shows that fine-tuning smaller models can outperform larger closed models like ChatGPT and GPT-4. The approach demonstrates the value of instruction-following and prompt-based reasoning for extracting KG knowledge from model parameters, with implications for scalable, text-aware KG completion. The authors also emphasize future improvements in prompt engineering and a two-stage ranking pipeline to further boost entity prediction accuracy.
Abstract
Knowledge graphs play a vital role in numerous artificial intelligence tasks, yet they frequently face the issue of incompleteness. In this study, we explore utilizing Large Language Models (LLM) for knowledge graph completion. We consider triples in knowledge graphs as text sequences and introduce an innovative framework called Knowledge Graph LLM (KG-LLM) to model these triples. Our technique employs entity and relation descriptions of a triple as prompts and utilizes the response for predictions. Experiments on various benchmark knowledge graphs demonstrate that our method attains state-of-the-art performance in tasks such as triple classification and relation prediction. We also find that fine-tuning relatively smaller models (e.g., LLaMA-7B, ChatGLM-6B) outperforms recent ChatGPT and GPT-4.