Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs

Kai Wang, Yuwei Xu, Zhiyong Wu, Siqiang Luo

TL;DR

This work tackles low-resource inductive reasoning on arbitrary knowledge graphs by pairing a pre-trained GNN reasoner with a frozen LLM-based prompter to generate a prompt graph that supplements sparse KGs without training. The ProLINK framework introduces role-aware relation encoding and a low-resource pretraining objective for the GNN, along with a prompt graph generation and calibration pipeline that leverages relation semantics and entity-type guidance from LLMs. Extensive experiments on 108 InGram-derived, low-resource KG datasets demonstrate that ProLINK consistently outperforms state-of-the-art baselines in 3-shot, 1-shot, and 0-shot settings, with notable gains when using GPT-4 and robust performance with Llama2 variants. The approach offers a scalable, training-free solution for cross-domain KG reasoning, though it relies on concise relational context and may benefit from richer semantic cues in future work.

Abstract

Knowledge Graph (KG) inductive reasoning, which aims to infer missing facts from new KGs that are not seen during training, has been widely adopted in various applications. One critical challenge of KG inductive reasoning is handling low-resource scenarios with scarcity in both textual and structural aspects. In this paper, we attempt to address this challenge with Large Language Models (LLMs). Particularly, we utilize the state-of-the-art LLMs to generate a graph-structural prompt to enhance the pre-trained Graph Neural Networks (GNNs), which brings us new methodological insights into the KG inductive reasoning methods, as well as high generalizability in practice. On the methodological side, we introduce a novel pretraining and prompting framework ProLINK, designed for low-resource inductive reasoning across arbitrary KGs without requiring additional training. On the practical side, we experimentally evaluate our approach on 36 low-resource KG datasets and find that ProLINK outperforms previous methods in three-shot, one-shot, and zero-shot reasoning tasks, exhibiting average performance improvements by 20%, 45%, and 147%, respectively. Furthermore, ProLINK demonstrates strong robustness for various LLM promptings as well as full-shot scenarios.

LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs

TL;DR

This work tackles low-resource inductive reasoning on arbitrary knowledge graphs by pairing a pre-trained GNN reasoner with a frozen LLM-based prompter to generate a prompt graph that supplements sparse KGs without training. The ProLINK framework introduces role-aware relation encoding and a low-resource pretraining objective for the GNN, along with a prompt graph generation and calibration pipeline that leverages relation semantics and entity-type guidance from LLMs. Extensive experiments on 108 InGram-derived, low-resource KG datasets demonstrate that ProLINK consistently outperforms state-of-the-art baselines in 3-shot, 1-shot, and 0-shot settings, with notable gains when using GPT-4 and robust performance with Llama2 variants. The approach offers a scalable, training-free solution for cross-domain KG reasoning, though it relies on concise relational context and may benefit from richer semantic cues in future work.

Abstract

Knowledge Graph (KG) inductive reasoning, which aims to infer missing facts from new KGs that are not seen during training, has been widely adopted in various applications. One critical challenge of KG inductive reasoning is handling low-resource scenarios with scarcity in both textual and structural aspects. In this paper, we attempt to address this challenge with Large Language Models (LLMs). Particularly, we utilize the state-of-the-art LLMs to generate a graph-structural prompt to enhance the pre-trained Graph Neural Networks (GNNs), which brings us new methodological insights into the KG inductive reasoning methods, as well as high generalizability in practice. On the methodological side, we introduce a novel pretraining and prompting framework ProLINK, designed for low-resource inductive reasoning across arbitrary KGs without requiring additional training. On the practical side, we experimentally evaluate our approach on 36 low-resource KG datasets and find that ProLINK outperforms previous methods in three-shot, one-shot, and zero-shot reasoning tasks, exhibiting average performance improvements by 20%, 45%, and 147%, respectively. Furthermore, ProLINK demonstrates strong robustness for various LLM promptings as well as full-shot scenarios.
Paper Structure (21 sections, 5 equations, 7 figures, 14 tables, 1 algorithm)

This paper contains 21 sections, 5 equations, 7 figures, 14 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Notations and Definitions
  4. Baseline: ULTRA
  5. Preliminaries: Low-resource Challenge
  6. Methodology
  7. GNN-based Reasoner
  8. LLM-based Prompter
  9. Prompt Calibrator
  10. Experiments
  11. Experimental Setup
  12. Main Experimental Results (RQ1)
  13. Performance of Different Prompts (RQ2)
  14. Ablation Studies (RQ3, RQ4)
  15. Conclusions
  16. ...and 6 more sections

Figures (7)

  • Figure 1: Illustrations of knowledge graphs and corresponding relation graphs. The interaction edge $h2t$ means the source relation has a head entity in the KG which is a tail entity of the target relation.
  • Figure 2: Preliminary results on ULTRA and Llama2.
  • Figure 3: ProLINK prompting process.
  • Figure 4: Examples of instruction prompts for LLMs.
  • Figure 5: Hits@10 results with different prompt settings in GPT-4. Darker colors indicate higher values.
  • ...and 2 more figures