Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Link Prediction with Untrained Message Passing Layers

Lisi Qarkaxhija, Anatol E. Wegner, Ingo Scholtes

TL;DR

This paper investigates link prediction on graphs with high-dimensional features using Untrained Message Passing (UTMP) layers, showing that removing trainable parameters from MP steps can yield competitive or superior performance while simplifying optimization. The authors derive and empirically validate connections between UTMP feature interactions and classical path-based similarity measures (e.g., Katz, rooted PageRank, Adamic-Adar), providing a theoretical lens on why UTMP features carry informative neighborhood structure. They study four widely used MP architectures (GCN, GraphSAGE, GIN, GraphConv) and their untrained counterparts (UTGCN, UTSAGE, UTGIN, UTGraphConv), including simplified variants with a final trained linear readout and fully untrained inner-product baselines. Across a broad suite of attributed and non-attributed graphs, UTMP variants often outperform their trained equivalents in link prediction and offer strong baselines with significantly reduced training costs, highlighting practical benefits for scalable graph learning and interpretation. The work also discusses extensions to other graph types (directed, weighted, temporal) and positions UTMP as a useful design principle for developing efficient, interpretable GNN architectures with clear ties to classical topological measures.

Abstract

Message passing neural networks (MPNNs) operate on graphs by exchanging information between neigbouring nodes. MPNNs have been successfully applied to various node-, edge-, and graph-level tasks in areas like molecular science, computer vision, natural language processing, and combinatorial optimization. However, most MPNNs require training on large amounts of labeled data, which can be costly and time-consuming. In this work, we explore the use of various untrained message passing layers in graph neural networks, i.e. variants of popular message passing architecture where we remove all trainable parameters that are used to transform node features in the message passing step. Focusing on link prediction, we find that untrained message passing layers can lead to competitive and even superior performance compared to fully trained MPNNs, especially in the presence of high-dimensional features. We provide a theoretical analysis of untrained message passing by relating the inner products of features implicitly produced by untrained message passing layers to path-based topological node similarity measures. As such, untrained message passing architectures can be viewed as a highly efficient and interpretable approach to link prediction.

Link Prediction with Untrained Message Passing Layers

TL;DR

This paper investigates link prediction on graphs with high-dimensional features using Untrained Message Passing (UTMP) layers, showing that removing trainable parameters from MP steps can yield competitive or superior performance while simplifying optimization. The authors derive and empirically validate connections between UTMP feature interactions and classical path-based similarity measures (e.g., Katz, rooted PageRank, Adamic-Adar), providing a theoretical lens on why UTMP features carry informative neighborhood structure. They study four widely used MP architectures (GCN, GraphSAGE, GIN, GraphConv) and their untrained counterparts (UTGCN, UTSAGE, UTGIN, UTGraphConv), including simplified variants with a final trained linear readout and fully untrained inner-product baselines. Across a broad suite of attributed and non-attributed graphs, UTMP variants often outperform their trained equivalents in link prediction and offer strong baselines with significantly reduced training costs, highlighting practical benefits for scalable graph learning and interpretation. The work also discusses extensions to other graph types (directed, weighted, temporal) and positions UTMP as a useful design principle for developing efficient, interpretable GNN architectures with clear ties to classical topological measures.

Abstract

Message passing neural networks (MPNNs) operate on graphs by exchanging information between neigbouring nodes. MPNNs have been successfully applied to various node-, edge-, and graph-level tasks in areas like molecular science, computer vision, natural language processing, and combinatorial optimization. However, most MPNNs require training on large amounts of labeled data, which can be costly and time-consuming. In this work, we explore the use of various untrained message passing layers in graph neural networks, i.e. variants of popular message passing architecture where we remove all trainable parameters that are used to transform node features in the message passing step. Focusing on link prediction, we find that untrained message passing layers can lead to competitive and even superior performance compared to fully trained MPNNs, especially in the presence of high-dimensional features. We provide a theoretical analysis of untrained message passing by relating the inner products of features implicitly produced by untrained message passing layers to path-based topological node similarity measures. As such, untrained message passing architectures can be viewed as a highly efficient and interpretable approach to link prediction.

Paper Structure

This paper contains 24 sections, 11 equations, 3 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Message Passing Architectures
  4. On the inclusion of self-loops
  5. Graph Convolutional Networks kipf2016semi
  6. GraphSAGE hamilton2017inductive
  7. GIN xu2018powerful
  8. GraphConv morris2019weisfeiler
  9. Untrained MP Architectures
  10. Simplified architectures
  11. UTMP layers and path based measures
  12. UTMP layers and path based measures
  13. Triadic closure and other path based measures
  14. Experiments and Results
  15. Experimental Setup
  16. ...and 9 more sections

Figures (3)

  • Figure 1: The distribution of feature dot products for pairs of connected and random node pairs for the attributed datasets.
  • Figure 2: The effect of increased layer size for fully untrained models.
  • Figure 3: Average runtimes (in seconds) for training and inference for attributed data sets.