A Relation-Interactive Approach for Message Passing in Hyper-relational Knowledge Graphs

TL;DR

This work proposes a message-passing-based graph encoder with global relation structure awareness ability, called ReSaE, which gives a encoding solution for hyper-relational KGs and ensures stronger performance on downstream link prediction tasks.

Abstract

Hyper-relational knowledge graphs (KGs) contain additional key-value pairs, providing more information about the relations. In many scenarios, the same relation can have distinct key-value pairs, making the original triple fact more recognizable and specific. Prior studies on hyper-relational KGs have established a solid standard method for hyper-relational graph encoding. In this work, we propose a message-passing-based graph encoder with global relation structure awareness ability, which we call ReSaE. Compared to the prior state-of-the-art approach, ReSaE emphasizes the interaction of relations during message passing process and optimizes the readout structure for link prediction tasks. Overall, ReSaE gives a encoding solution for hyper-relational KGs and ensures stronger performance on downstream link prediction tasks. Our experiments demonstrate that ReSaE achieves state-of-the-art performance on multiple link prediction benchmarks. Furthermore, we also analyze the influence of different model structures on model performance.

Figures (5)

• Figure 1: Two examples on hyper-relational facts. The left fact is (George Miller , nominated for, Academy Award for Best Animated Feature) with qualifier (for work, Happy Feet); The right fact is (George Miller, nominated for, Academy Award for Best Pictures) with qualifier (for work, Babe).
• Figure 2: The self-attention illustration: On the left, we have the similarity score matrix for all relation set in a hyper-relational KG. Then for each fact, the qualifier relation weights are scattered regarding the main fact relation.
• Figure 3: the hyper-relational fact entity update process. $h_{hyper}$ contains information of fact entity, main relation, qualifier entities aggregation, qualifier relations aggregation, aggregation of $att(h_{qr})$. $h_v^\gamma$ is linear projection of $h_{hyper}$ aggregation output and $f$ denotes the activation function.
• Figure 4: The relation update process: On the top left, each cell represents the co-occurrence time of two relations in all hyper-relational facts. We then normalize each cell by dividing it by mean of row sum, resulting in the matrix on the top right. Each type (relation direction) of relations is updated by weight sum of itself and linear projection of (relation * coo-occurrence matrix). Finally, the relation representation is updated by taking the average of its type.
• Figure 5: The decoder architecture. Hyper-relational fact statements go through transformer layers. Subsequently, the output sequence is pooled based on their type. The final hidden state is attained through a fully connected layer.