Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Knowledge Graph Completion with GNN Distillation and Probabilistic Interaction Modeling

Lingzhi Wang, Pengcheng Huang, Haotian Li, Yuliang Wei, Guodong Xin, Rui Zhang, Donglin Zhang, Zhenzhou Ji, Wei Wang

TL;DR

This work addresses knowledge graph completion by tackling over-smoothing in deep GNNs and the absence of high-level relational abstractions in embeddings. It introduces GNN distillation to iteratively filter messages and APIM to learn probabilistic interaction patterns, enabling enhanced local discriminative power and global relational modeling. The approach is integrated into both GNN-based and embedding-based KGC models, with extensive experiments on WN18RR and FB15K-237 showing significant and synergistic improvements, and ablations identifying effective hyperparameters such as Top-K and distillation decay schedules. The proposed framework provides a practical, extensible path toward more accurate and robust KGC systems with implications for downstream reasoning and data integration.

Abstract

Knowledge graphs (KGs) serve as fundamental structures for organizing interconnected data across diverse domains. However, most KGs remain incomplete, limiting their effectiveness in downstream applications. Knowledge graph completion (KGC) aims to address this issue by inferring missing links, but existing methods face critical challenges: deep graph neural networks (GNNs) suffer from over-smoothing, while embedding-based models fail to capture abstract relational features. This study aims to overcome these limitations by proposing a unified framework that integrates GNN distillation and abstract probabilistic interaction modeling (APIM). GNN distillation approach introduces an iterative message-feature filtering process to mitigate over-smoothing, preserving the discriminative power of node representations. APIM module complements this by learning structured, abstract interaction patterns through probabilistic signatures and transition matrices, allowing for a richer, more flexible representation of entity and relation interactions. We apply these methods to GNN-based models and the APIM to embedding-based KGC models, conducting extensive evaluations on the widely used WN18RR and FB15K-237 datasets. Our results demonstrate significant performance gains over baseline models, showcasing the effectiveness of the proposed techniques. The findings highlight the importance of both controlling information propagation and leveraging structured probabilistic modeling, offering new avenues for advancing knowledge graph completion. And our codes are available at https://anonymous.4open.science/r/APIM_and_GNN-Distillation-461C.

Enhancing Knowledge Graph Completion with GNN Distillation and Probabilistic Interaction Modeling

TL;DR

This work addresses knowledge graph completion by tackling over-smoothing in deep GNNs and the absence of high-level relational abstractions in embeddings. It introduces GNN distillation to iteratively filter messages and APIM to learn probabilistic interaction patterns, enabling enhanced local discriminative power and global relational modeling. The approach is integrated into both GNN-based and embedding-based KGC models, with extensive experiments on WN18RR and FB15K-237 showing significant and synergistic improvements, and ablations identifying effective hyperparameters such as Top-K and distillation decay schedules. The proposed framework provides a practical, extensible path toward more accurate and robust KGC systems with implications for downstream reasoning and data integration.

Abstract

Knowledge graphs (KGs) serve as fundamental structures for organizing interconnected data across diverse domains. However, most KGs remain incomplete, limiting their effectiveness in downstream applications. Knowledge graph completion (KGC) aims to address this issue by inferring missing links, but existing methods face critical challenges: deep graph neural networks (GNNs) suffer from over-smoothing, while embedding-based models fail to capture abstract relational features. This study aims to overcome these limitations by proposing a unified framework that integrates GNN distillation and abstract probabilistic interaction modeling (APIM). GNN distillation approach introduces an iterative message-feature filtering process to mitigate over-smoothing, preserving the discriminative power of node representations. APIM module complements this by learning structured, abstract interaction patterns through probabilistic signatures and transition matrices, allowing for a richer, more flexible representation of entity and relation interactions. We apply these methods to GNN-based models and the APIM to embedding-based KGC models, conducting extensive evaluations on the widely used WN18RR and FB15K-237 datasets. Our results demonstrate significant performance gains over baseline models, showcasing the effectiveness of the proposed techniques. The findings highlight the importance of both controlling information propagation and leveraging structured probabilistic modeling, offering new avenues for advancing knowledge graph completion. And our codes are available at https://anonymous.4open.science/r/APIM_and_GNN-Distillation-461C.
Paper Structure (20 sections, 16 equations, 7 figures, 2 tables)

This paper contains 20 sections, 16 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Knowledge graph completion (KGC)
  4. GNN-based KGC methods
  5. Emebdding-based KGC methods
  6. Methodologys
  7. GNN distillation method
  8. The abstract probabilistic interaction modeling (APIM) method
  9. The GNN-based models enhancement
  10. The Embedding-based models enhancement
  11. Experiments
  12. Experiments setup
  13. Main results and analysis
  14. Ablation experiments
  15. Discussion
  16. ...and 5 more sections

Figures (7)

  • Figure 1: A general structure of the MPNNs-based KGC methods.
  • Figure 2: The visualisation of the GNN Distillation method.
  • Figure 3: The enhanced KB-GAT framework integrates a GNN distillation module that refines message representations by combining them with attention weights $\alpha$ and an APIM module that models abstract interaction patterns. In the diagram, $\otimes$ denotes matrix multiplication, while $\odot$ represents the GCN aggregation operation.
  • Figure 4: Impact of APIM's Top-K mode retention thresholds on KGC performance (WN18RR)
  • Figure 5: Impact of GNN Distillation's filtering ratio on KB-GAT performance (FB15K-237)
  • ...and 2 more figures