Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReFactor GNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective

Yihong Chen, Pushkar Mishra, Luca Franceschi, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel

TL;DR

This work investigates bridging factorisation-based models (FMs) and graph neural networks (GNNs) for multi-relational link prediction in knowledge graphs. By reformulating FM training dynamics as message-passing updates, the authors introduce ReFactor GNNs, a family of architectures that interpolate between FM and GNN paradigms and support inductive reasoning with node features while maintaining parameter efficiency. Empirically, ReFactor GNNs achieve state-of-the-art inductive performance and competitive transductive performance across multiple benchmarks, using an order of magnitude fewer parameters. A key contribution is the augmented message-passing component $n[v]$, which captures global information and enhances generalisation. The results offer a practical and scalable pathway to combine FM accuracy with GNN inductive capabilities for knowledge graph completion.

Abstract

Factorisation-based Models (FMs), such as DistMult, have enjoyed enduring success for Knowledge Graph Completion (KGC) tasks, often outperforming Graph Neural Networks (GNNs). However, unlike GNNs, FMs struggle to incorporate node features and generalise to unseen nodes in inductive settings. Our work bridges the gap between FMs and GNNs by proposing ReFactor GNNs. This new architecture draws upon both modelling paradigms, which previously were largely thought of as disjoint. Concretely, using a message-passing formalism, we show how FMs can be cast as GNNs by reformulating the gradient descent procedure as message-passing operations, which forms the basis of our ReFactor GNNs. Across a multitude of well-established KGC benchmarks, our ReFactor GNNs achieve comparable transductive performance to FMs, and state-of-the-art inductive performance while using an order of magnitude fewer parameters.

ReFactor GNNs: Revisiting Factorisation-based Models from a Message-Passing Perspective

TL;DR

This work investigates bridging factorisation-based models (FMs) and graph neural networks (GNNs) for multi-relational link prediction in knowledge graphs. By reformulating FM training dynamics as message-passing updates, the authors introduce ReFactor GNNs, a family of architectures that interpolate between FM and GNN paradigms and support inductive reasoning with node features while maintaining parameter efficiency. Empirically, ReFactor GNNs achieve state-of-the-art inductive performance and competitive transductive performance across multiple benchmarks, using an order of magnitude fewer parameters. A key contribution is the augmented message-passing component , which captures global information and enhances generalisation. The results offer a practical and scalable pathway to combine FM accuracy with GNN inductive capabilities for knowledge graph completion.

Abstract

Factorisation-based Models (FMs), such as DistMult, have enjoyed enduring success for Knowledge Graph Completion (KGC) tasks, often outperforming Graph Neural Networks (GNNs). However, unlike GNNs, FMs struggle to incorporate node features and generalise to unseen nodes in inductive settings. Our work bridges the gap between FMs and GNNs by proposing ReFactor GNNs. This new architecture draws upon both modelling paradigms, which previously were largely thought of as disjoint. Concretely, using a message-passing formalism, we show how FMs can be cast as GNNs by reformulating the gradient descent procedure as message-passing operations, which forms the basis of our ReFactor GNNs. Across a multitude of well-established KGC benchmarks, our ReFactor GNNs achieve comparable transductive performance to FMs, and state-of-the-art inductive performance while using an order of magnitude fewer parameters.
Paper Structure (35 sections, 3 theorems, 36 equations, 4 figures, 4 tables)

This paper contains 35 sections, 3 theorems, 36 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Factorisation-based Models for KGC
  4. GNN-based Models for KGC
  5. Implicit Message-Passing in FMs
  6. The Edge View
  7. The Node View
  8. ReFactor GNNs
  9. The ReFactor Layer
  10. Training
  11. Experiments
  12. ReFactor GNNs for Transductive Learning (Q1)
  13. ReFactor GNNs for Inductive Learning (Q2)
  14. Performance vs Parameter Efficiency as #Message-Passing Layers Increases
  15. Beyond Message-Passing (Q3)
  16. ...and 20 more sections

Key Result

Theorem 3.1

The gradient descent operator $\mathop{\mathrm{GD}}\nolimits$ (eq:fm_node_view) on the node embeddings of a DistMult model (eq:fm_entity_encoder) with the maximum likelihood objective in eq:objective and a multi-relational graph $\mathcal{T}$ defined over entities $\mathcal{E}$ induces a message-pas where, defining the sets of triplets $\mathcal{T}^{-v} = \{ (s, r, o) \in \mathcal{T} \; : \; s\neq

Figures (4)

  • Figure 1: ReFactor GNN architecture -- the left figure describes the messages (coloured edges) used to update the representation of node $v_{1}$, which depend on the type of relationship between the sender nodes and $v_{1}$ in the graph $G = \{ (v_2, r_1, v_1), (v_3, r_2, v_1), (v_1, r_3, v_4) \}$; the right figure describes the computation graph for calculating $P(v \mid w, r)$, where $v, w \in \mathcal{E}$ and $r \in \mathcal{R}$: the embedding representations of $w$, $r$, and $v$ are used to score the edge $(w, r, v)$ via the scoring function $\Gamma$, which is then normalised via the $\text{Softmax}$ function.
  • Figure 2: Inductive KGC Performance. Models are trained on the KG FB15K237_v1 and tested on another KG FB15K237_v1_ind, where the entities are completely new. The results of GraIL and NBFNet are taken from zhaochengzhu2021. The grey bars indicate methods that are not devised to incorporate node features.
  • Figure 3: Performance vs Parameter Efficiency as #Layers Increases on FB15K237_v1. The left axis is Test MRR while the right axis is #Parameters. The solid lines and dashed lines indicate the changes of Test MRR and the changes of #Parameters.
  • Figure 4: Performance vs Parameter Efficiency as #Layers Increases on FB15K237_v2. The left axis is Test MRR while the right axis is #Parameters. The solid lines and dashed lines indicate the changes of Test MRR and the changes of #Parameters.

Theorems & Definitions (4)

  • Theorem 3.1: Message passing in FMs
  • Theorem A.1: Message passing in FMs
  • proof
  • Lemma A.2: Message passing in FMs