Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Retrieval, Reasoning, Re-ranking: A Context-Enriched Framework for Knowledge Graph Completion

Muzhi Li, Cehao Yang, Chengjin Xu, Xuhui Jiang, Yiyan Qi, Jian Guo, Ho-fung Leung, Irwin King

TL;DR

Knowledge graph completion is hampered by the semantic gap between graph structure and natural language, especially for long-tail entities. The authors introduce KGR^3, a context-enriched three-stage framework (Retrieval, Reasoning, Re-ranking) that retrieves supporting triples and textual contexts, uses in-context demonstrations for reasoning with an LLM, and fine-tunes a re-ranker to integrate base KGC scores with LLM-derived candidates. The approach consistently improves performance across multiple base KGC models on FB15k237 and WN18RR, achieving state-of-the-art gains, notably $Hits@1$ improvements of $12.3\%$ and $5.6\%$ respectively. This demonstrates the value of integrating structured knowledge, entity contexts, and LLM-based reasoning for robust and scalable KG completion.

Abstract

The Knowledge Graph Completion~(KGC) task aims to infer the missing entity from an incomplete triple. Existing embedding-based methods rely solely on triples in the KG, which is vulnerable to specious relation patterns and long-tail entities. On the other hand, text-based methods struggle with the semantic gap between KG triples and natural language. Apart from triples, entity contexts (e.g., labels, descriptions, aliases) also play a significant role in augmenting KGs. To address these limitations, we propose KGR3, a context-enriched framework for KGC. KGR3 is composed of three modules. Firstly, the Retrieval module gathers supporting triples from the KG, collects plausible candidate answers from a base embedding model, and retrieves context for each related entity. Then, the Reasoning module employs a large language model to generate potential answers for each query triple. Finally, the Re-ranking module combines candidate answers from the two modules mentioned above, and fine-tunes an LLM to provide the best answer. Extensive experiments on widely used datasets demonstrate that KGR3 consistently improves various KGC methods. Specifically, the best variant of KGR3 achieves absolute Hits@1 improvements of 12.3% and 5.6% on the FB15k237 and WN18RR datasets.

Retrieval, Reasoning, Re-ranking: A Context-Enriched Framework for Knowledge Graph Completion

TL;DR

Knowledge graph completion is hampered by the semantic gap between graph structure and natural language, especially for long-tail entities. The authors introduce KGR^3, a context-enriched three-stage framework (Retrieval, Reasoning, Re-ranking) that retrieves supporting triples and textual contexts, uses in-context demonstrations for reasoning with an LLM, and fine-tunes a re-ranker to integrate base KGC scores with LLM-derived candidates. The approach consistently improves performance across multiple base KGC models on FB15k237 and WN18RR, achieving state-of-the-art gains, notably improvements of and respectively. This demonstrates the value of integrating structured knowledge, entity contexts, and LLM-based reasoning for robust and scalable KG completion.

Abstract

The Knowledge Graph Completion~(KGC) task aims to infer the missing entity from an incomplete triple. Existing embedding-based methods rely solely on triples in the KG, which is vulnerable to specious relation patterns and long-tail entities. On the other hand, text-based methods struggle with the semantic gap between KG triples and natural language. Apart from triples, entity contexts (e.g., labels, descriptions, aliases) also play a significant role in augmenting KGs. To address these limitations, we propose KGR3, a context-enriched framework for KGC. KGR3 is composed of three modules. Firstly, the Retrieval module gathers supporting triples from the KG, collects plausible candidate answers from a base embedding model, and retrieves context for each related entity. Then, the Reasoning module employs a large language model to generate potential answers for each query triple. Finally, the Re-ranking module combines candidate answers from the two modules mentioned above, and fine-tunes an LLM to provide the best answer. Extensive experiments on widely used datasets demonstrate that KGR3 consistently improves various KGC methods. Specifically, the best variant of KGR3 achieves absolute Hits@1 improvements of 12.3% and 5.6% on the FB15k237 and WN18RR datasets.

Paper Structure

This paper contains 31 sections, 5 equations, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Embedding-based methods
  4. Text-based methods.
  5. Problem Specifications
  6. Methods
  7. Stage 1: Retrieval
  8. Supporting Triple Retrieval
  9. Textual Context Retrieval
  10. Candidate Answer Retrieval
  11. Stage 2: Reasoning
  12. Supporting Triple Demonstrations
  13. Context-aware Reasoning
  14. Stage 3: Re-ranking
  15. Experiments
  16. ...and 16 more sections

Figures (3)

  • Figure 1: Limitations of existing embedding-based (top) and LLM-based (bottom) KGC methods.
  • Figure 2: The end-to-end architecture of the proposed $\text{KGR}^3$ framework, which consists of three modules: 1) Reasoning (left), 2) Reasoning (middle), and 3) Re-ranking (right). (Please see Appendix \ref{['prompts']} for detailed prompts.)
  • Figure 3: Hits@$1$ performance over the amount of SFT data on the two datasets with different LLMs.