THOR: Inductive Link Prediction over Hyper-Relational Knowledge Graphs

TL;DR

THOR tackles inductive link prediction on hyper-relational knowledge graphs by introducing relation and entity foundation graphs that abstract away specific vocabularies. It employs two parallel Neural Bellman-Ford Network encoders on these foundations, followed by a transformer decoder to perform masked inference without negative sampling. The approach achieves substantial gains across 12 datasets, including cross-domain fully-inductive settings, and ablation studies identify the fundamental interactions that transfer across HKGs. This work advances scalable, generalizable reasoning on hyper-relational facts and offers a blueprint for foundation-model-style KG reasoning.

Abstract

Knowledge graphs (KGs) have become a key ingredient supporting a variety of applications. Beyond the traditional triplet representation of facts where a relation connects two entities, modern KGs observe an increasing number of hyper-relational facts, where an arbitrary number of qualifiers associated with a triplet provide auxiliary information to further describe the rich semantics of the triplet, which can effectively boost the reasoning performance in link prediction tasks. However, existing link prediction techniques over such hyper-relational KGs (HKGs) mostly focus on a transductive setting, where KG embedding models are learned from the specific vocabulary of a given KG and subsequently can only make predictions within the same vocabulary, limiting their generalizability to previously unseen vocabularies. Against this background, we propose THOR, an inducTive link prediction technique for Hyper-relational knOwledge gRaphs. Specifically, we first introduce both relation and entity foundation graphs, modeling their fundamental inter- and intra-fact interactions in HKGs, which are agnostic to any specific relations and entities. Afterward, THOR is designed to learn from the two foundation graphs with two parallel graph encoders followed by a transformer decoder, which supports efficient masked training and fully-inductive inference. We conduct a thorough evaluation of THOR in hyper-relational link prediction tasks on 12 datasets with different settings. Results show that THOR outperforms a sizable collection of baselines, yielding 66.1%, 55.9%, and 20.4% improvement over the best-performing rule-based, semi-inductive, and fully-inductive techniques, respectively. A series of ablation studies also reveals our key design factors capturing the structural invariance transferable across HKGs for inductive tasks.

Figures (3)

• Figure 1: Relation foundation graph with different fundamental interactions. The plot focuses on the relations of the fact highlighted in red, illustrating all the fundamental interactions involving its relations $r_1$, $k_1$, and $k_2$ only. Interactions among other relations also exist (e.g., $h2t_r$ and $t2h_r$ interactions between $r_2$ and $r_3$), but are not shown for the clarity of visualization.
• Figure 2: Entity foundation graph with different types of fundamental intra-fact interactions.
• Figure 3: Architectural overview of THOR with 1) a relation foundation graph encoder, 2) an entity foundation graph encoder, and 3) a transformer-based decoder.