Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Relation-aware Neighborhood Aggregation in Graph Neural Networks via Tensor Decomposition

Peyman Baghershahi, Reshad Hosseini, Hadi Moradi

TL;DR

A novel knowledge graph encoder that incorporates tensor decomposition within the aggregation function of Relational Graph Convolutional Network (R-GCN), which enhances the representation of neighboring entities by employing projection matrices of a low-rank tensor defined by relation types.

Abstract

Numerous Graph Neural Networks (GNNs) have been developed to tackle the challenge of Knowledge Graph Embedding (KGE). However, many of these approaches overlook the crucial role of relation information and inadequately integrate it with entity information, resulting in diminished expressive power. In this paper, we propose a novel knowledge graph encoder that incorporates tensor decomposition within the aggregation function of Relational Graph Convolutional Network (R-GCN). Our model enhances the representation of neighboring entities by employing projection matrices of a low-rank tensor defined by relation types. This approach facilitates multi-task learning, thereby generating relation-aware representations. Furthermore, we introduce a low-rank estimation technique for the core tensor through CP decomposition, which effectively compresses and regularizes our model. We adopt a training strategy inspired by contrastive learning, which relieves the training limitation of the 1-N method inherent in handling vast graphs. We outperformed all our competitors on two common benchmark datasets, FB15k-237 and WN18RR, while using low-dimensional embeddings for entities and relations.

Efficient Relation-aware Neighborhood Aggregation in Graph Neural Networks via Tensor Decomposition

TL;DR

A novel knowledge graph encoder that incorporates tensor decomposition within the aggregation function of Relational Graph Convolutional Network (R-GCN), which enhances the representation of neighboring entities by employing projection matrices of a low-rank tensor defined by relation types.

Abstract

Numerous Graph Neural Networks (GNNs) have been developed to tackle the challenge of Knowledge Graph Embedding (KGE). However, many of these approaches overlook the crucial role of relation information and inadequately integrate it with entity information, resulting in diminished expressive power. In this paper, we propose a novel knowledge graph encoder that incorporates tensor decomposition within the aggregation function of Relational Graph Convolutional Network (R-GCN). Our model enhances the representation of neighboring entities by employing projection matrices of a low-rank tensor defined by relation types. This approach facilitates multi-task learning, thereby generating relation-aware representations. Furthermore, we introduce a low-rank estimation technique for the core tensor through CP decomposition, which effectively compresses and regularizes our model. We adopt a training strategy inspired by contrastive learning, which relieves the training limitation of the 1-N method inherent in handling vast graphs. We outperformed all our competitors on two common benchmark datasets, FB15k-237 and WN18RR, while using low-dimensional embeddings for entities and relations.
Paper Structure (27 sections, 11 equations, 2 figures, 6 tables)

This paper contains 27 sections, 11 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Contributions:
  3. Related Work
  4. Knowledge Graph Embedding (KGE)
  5. Graph Neural Networks (GNNs)
  6. GNNs in KGE
  7. Background
  8. Problem
  9. Methods
  10. Model Compression
  11. Training
  12. 1-b Training
  13. Experiments
  14. Using Random Subgraphs
  15. Datasets
  16. ...and 12 more sections

Figures (2)

  • Figure 1: The effect of the number of bases ($n_b$) for CP decomposition. The green line shows the performance of TGCN-Tucker on WN18RR for $d=100$ with different $n_b$ and the blue line shows its #ENFP. Both figures are on a logarithmic scale, meaning that #ENFP increases linearly by increasing $n_b$ although its curve it if exponential.
  • Figure 2: The effect of the size of the random subgraphs ($g_s$) for training on the performance of TGCN-Tucker on FB15k-237 and WN188.