Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MUSE: Multi-Knowledge Passing on the Edges, Boosting Knowledge Graph Completion

Pengjie Liu

TL;DR

Knowledge Graph Completion struggles with incomplete triples and long-tail nodes. MUSE introduces a triadic embedding approach with Prior Knowledge Learning (BERT fine-tuning on entity descriptions), Context Message Passing (edge-focused, attention-guided message flow), and Relational Path Aggregation (path-level attention), unified via a Multi-Knowledge Fusion mechanism. It achieves state-of-the-art results across FB15k-237, WN18, WN18RR, and NELL995, with notable gains under LIS conditions, and ablation confirms the value of each module and the fine-tuned semantic signals. The work advances KG completion by effectively injecting external semantic knowledge into graph-based reasoning and provides reproducible code for broader adoption.

Abstract

Knowledge Graph Completion (KGC) aims to predict the missing information in the (head entity)-[relation]-(tail entity) triplet. Deep Neural Networks have achieved significant progress in the relation prediction task. However, most existing KGC methods focus on single features (e.g., entity IDs) and sub-graph aggregation, which cannot fully explore all the features in the Knowledge Graph (KG), and neglect the external semantic knowledge injection. To address these problems, we propose MUSE, a knowledge-aware reasoning model to learn a tailored embedding space in three dimensions for missing relation prediction through a multi-knowledge representation learning mechanism. Our MUSE consists of three parallel components: 1) Prior Knowledge Learning for enhancing the triplets' semantic representation by fine-tuning BERT; 2) Context Message Passing for enhancing the context messages of KG; 3) Relational Path Aggregation for enhancing the path representation from the head entity to the tail entity. Our experimental results show that MUSE significantly outperforms other baselines on four public datasets, such as over 5.50% improvement in H@1 and 4.20% improvement in MRR on the NELL995 dataset. The code and all datasets will be released via https://github.com/NxxTGT/MUSE.

MUSE: Multi-Knowledge Passing on the Edges, Boosting Knowledge Graph Completion

TL;DR

Knowledge Graph Completion struggles with incomplete triples and long-tail nodes. MUSE introduces a triadic embedding approach with Prior Knowledge Learning (BERT fine-tuning on entity descriptions), Context Message Passing (edge-focused, attention-guided message flow), and Relational Path Aggregation (path-level attention), unified via a Multi-Knowledge Fusion mechanism. It achieves state-of-the-art results across FB15k-237, WN18, WN18RR, and NELL995, with notable gains under LIS conditions, and ablation confirms the value of each module and the fine-tuned semantic signals. The work advances KG completion by effectively injecting external semantic knowledge into graph-based reasoning and provides reproducible code for broader adoption.

Abstract

Knowledge Graph Completion (KGC) aims to predict the missing information in the (head entity)-[relation]-(tail entity) triplet. Deep Neural Networks have achieved significant progress in the relation prediction task. However, most existing KGC methods focus on single features (e.g., entity IDs) and sub-graph aggregation, which cannot fully explore all the features in the Knowledge Graph (KG), and neglect the external semantic knowledge injection. To address these problems, we propose MUSE, a knowledge-aware reasoning model to learn a tailored embedding space in three dimensions for missing relation prediction through a multi-knowledge representation learning mechanism. Our MUSE consists of three parallel components: 1) Prior Knowledge Learning for enhancing the triplets' semantic representation by fine-tuning BERT; 2) Context Message Passing for enhancing the context messages of KG; 3) Relational Path Aggregation for enhancing the path representation from the head entity to the tail entity. Our experimental results show that MUSE significantly outperforms other baselines on four public datasets, such as over 5.50% improvement in H@1 and 4.20% improvement in MRR on the NELL995 dataset. The code and all datasets will be released via https://github.com/NxxTGT/MUSE.
Paper Structure (16 sections, 12 equations, 5 figures, 3 tables)

This paper contains 16 sections, 12 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Preliminary of MUSE
  4. Prior Knowledge Learning.
  5. Context Message Passing.
  6. Relational Path Aggregation.
  7. Multi-Knowledge Fusion
  8. Experimental Methodology
  9. Datasets
  10. Baselines
  11. Implementation Details
  12. Evaluation Results
  13. Overall Performance
  14. Ablation Study
  15. Contributions of Multi-Knowledge in MUSE
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Two Example Cases of Relation Prediction. Entity degree is the max of the sum of the out-degree and in-degree, or the number of paths from this entity to the connected entities. $\text{Entity Degree} = {max\{\text{(in-degree + out-degree), paths\}}}$.
  • Figure 2: The Architecture of MUSE Framework.
  • Figure 3: Illustration of the Prior Knowledge Learning. We fine-tune BERT on FB15k-237, WN18, WN18RR, and NELL995, respectively.
  • Figure 4: Analysis of the External Semantic Knowledge in the Relation Prediction Task. We define the $\text{MUSE~w/o~Fine-Tuning}$ model as applying the BERT model directly in Figure \ref{['fig:pkla']}. As shown in Figure \ref{['fig:pklb']}, the $\text{MUSE w/o Attention}$ model aggregates entities without using attention mechanism in Equation \ref{['eq:attention']}.
  • Figure 5: Performance of MUSE and PathCon on the NELL995.