Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HYPER: A Foundation Model for Inductive Link Prediction with Knowledge Hypergraphs

Xingyue Huang, Mikhail Galkin, Michael M. Bronstein, İsmail İlkan Ceylan

TL;DR

HYPER is proposed as a foundation model for link prediction, which can generalize to any knowledge hypergraph, including novel entities and novel relations, and consistently outperforms all existing methods in both node-only and node-and-relation inductive settings.

Abstract

Inductive link prediction with knowledge hypergraphs is the task of predicting missing hyperedges involving completely novel entities (i.e., nodes unseen during training). Existing methods for inductive link prediction with knowledge hypergraphs assume a fixed relational vocabulary and, as a result, cannot generalize to knowledge hypergraphs with novel relation types (i.e., relations unseen during training). Inspired by knowledge graph foundation models, we propose HYPER as a foundation model for link prediction, which can generalize to any knowledge hypergraph, including novel entities and novel relations. Importantly, HYPER can learn and transfer across different relation types of varying arities, by encoding the entities of each hyperedge along with their respective positions in the hyperedge. To evaluate HYPER, we construct 16 new inductive datasets from existing knowledge hypergraphs, covering a diverse range of relation types of varying arities. Empirically, HYPER consistently outperforms all existing methods in both node-only and node-and-relation inductive settings, showing strong generalization to unseen, higher-arity relational structures.

HYPER: A Foundation Model for Inductive Link Prediction with Knowledge Hypergraphs

TL;DR

HYPER is proposed as a foundation model for link prediction, which can generalize to any knowledge hypergraph, including novel entities and novel relations, and consistently outperforms all existing methods in both node-only and node-and-relation inductive settings.

Abstract

Inductive link prediction with knowledge hypergraphs is the task of predicting missing hyperedges involving completely novel entities (i.e., nodes unseen during training). Existing methods for inductive link prediction with knowledge hypergraphs assume a fixed relational vocabulary and, as a result, cannot generalize to knowledge hypergraphs with novel relation types (i.e., relations unseen during training). Inspired by knowledge graph foundation models, we propose HYPER as a foundation model for link prediction, which can generalize to any knowledge hypergraph, including novel entities and novel relations. Importantly, HYPER can learn and transfer across different relation types of varying arities, by encoding the entities of each hyperedge along with their respective positions in the hyperedge. To evaluate HYPER, we construct 16 new inductive datasets from existing knowledge hypergraphs, covering a diverse range of relation types of varying arities. Empirically, HYPER consistently outperforms all existing methods in both node-only and node-and-relation inductive settings, showing strong generalization to unseen, higher-arity relational structures.

Paper Structure

This paper contains 50 sections, 3 theorems, 35 equations, 8 figures, 23 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Preliminaries
  5. Knowledge Hypergraphs.
  6. Link Prediction on Hyperedges.
  7. Reification.
  8. Link Prediction over Reified Knowledge Hypergraphs.
  9. Hyper: A Knowledge Hypergraph Foundation Model
  10. Encoding the relations.
  11. Entity encoder.
  12. Experiments
  13. Experimental Setups
  14. Node-Relation Inductive Link Prediction over Knowledge Hypergraphs
  15. Node Inductive Link Prediction over Knowledge Hypergraphs
  16. ...and 35 more sections

Key Result

Theorem 4.1

There exists a set of parameter for ${\textsc{Hyper}}$ such that $\textsf{Enc}\xspace_{\mathrm{PI}}$ is injective, has a bounded range, and is Lipschitz (and hence locally smooth).

Figures (8)

  • Figure 1: A knowledge hypergraph with three hyperedges over distinct relation types.
  • Figure 2: A model is trained on relations like $\mathsf{Research}$, $\mathsf{Teaches}$, and $\mathsf{AtConference}$, and is expected to generalize to structurally similar relations $\mathsf{TradingDeal}$, $\mathsf{Sells}$, and $\mathsf{AtBusinessFair}$ at test time.
  • Figure 3: The relation graph $G_{\text{rel}}$ corresponding to the knowledge hypergraph $G_{\text{train}}$.
  • Figure 4: Reified KG corresponding to the knowledge hypergraph $G_\text{train}$ from Fig \ref{['fig:knowledge-hypergraph']}. $\mathsf{R}$-$\mathsf{i}$ abbreviates $\mathsf{Research}$-$\mathsf{i}$, similarly for $\mathsf{A}$ as $\mathsf{AtConference}$, and $\mathsf{T}$ as $\mathsf{Teaches}$.
  • Figure 4: Averaged zero-shot performance of ${\textsc{Hyper}}$(3KG + 2HG) with different positional interaction encoders.
  • ...and 3 more figures

Theorems & Definitions (5)

  • Theorem 4.1: Informal
  • Proposition C.1: invariance
  • proof
  • Theorem C.2: Properties of the positional–interaction encoder
  • proof