Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Positional and Structural Encoder

Semih Cantürk, Renming Liu, Olivier Lapointe-Gagné, Vincent Létourneau, Guy Wolf, Dominique Beaini, Ladislav Rampášek

TL;DR

The paper tackles the lack of a universal, task-agnostic positional and structural encoding (PSE) for graphs by introducing GPSE, a foundation encoder trained to reconstruct a diverse set of PSEs from graph structure alone. GPSE uses a deep MPNN with random node features, residual gating, and a virtual node to learn a common latent representation that can augment any graph neural model, including graph Transformers, across distributions and modalities. It demonstrates strong PSE recovery (up to $R^2\approx$0.998) and delivers state-of-the-art or competitive improvements on molecular benchmarks (e.g., ZINC, PCQM4Mv2, MolHIV, MolPCBA) as well as robust transferability to non-molecular domains and OOD scenarios. The work argues that GPSE provides a scalable alternative to hand-crafted PSEs and SSL pre-training, enabling more powerful, generalizable encodings that can underpin future foundation models for graph learning, with public code and PyG integration to ease adoption.

Abstract

Positional and structural encodings (PSE) enable better identifiability of nodes within a graph, rendering them essential tools for empowering modern GNNs, and in particular graph Transformers. However, designing PSEs that work optimally for all graph prediction tasks is a challenging and unsolved problem. Here, we present the Graph Positional and Structural Encoder (GPSE), the first-ever graph encoder designed to capture rich PSE representations for augmenting any GNN. GPSE learns an efficient common latent representation for multiple PSEs, and is highly transferable: The encoder trained on a particular graph dataset can be used effectively on datasets drawn from markedly different distributions and modalities. We show that across a wide range of benchmarks, GPSE-enhanced models can significantly outperform those that employ explicitly computed PSEs, and at least match their performance in others. Our results pave the way for the development of foundational pre-trained graph encoders for extracting positional and structural information, and highlight their potential as a more powerful and efficient alternative to explicitly computed PSEs and existing self-supervised pre-training approaches. Our framework and pre-trained models are publicly available at https://github.com/G-Taxonomy-Workgroup/GPSE. For convenience, GPSE has also been integrated into the PyG library to facilitate downstream applications.

Graph Positional and Structural Encoder

TL;DR

The paper tackles the lack of a universal, task-agnostic positional and structural encoding (PSE) for graphs by introducing GPSE, a foundation encoder trained to reconstruct a diverse set of PSEs from graph structure alone. GPSE uses a deep MPNN with random node features, residual gating, and a virtual node to learn a common latent representation that can augment any graph neural model, including graph Transformers, across distributions and modalities. It demonstrates strong PSE recovery (up to 0.998) and delivers state-of-the-art or competitive improvements on molecular benchmarks (e.g., ZINC, PCQM4Mv2, MolHIV, MolPCBA) as well as robust transferability to non-molecular domains and OOD scenarios. The work argues that GPSE provides a scalable alternative to hand-crafted PSEs and SSL pre-training, enabling more powerful, generalizable encodings that can underpin future foundation models for graph learning, with public code and PyG integration to ease adoption.

Abstract

Positional and structural encodings (PSE) enable better identifiability of nodes within a graph, rendering them essential tools for empowering modern GNNs, and in particular graph Transformers. However, designing PSEs that work optimally for all graph prediction tasks is a challenging and unsolved problem. Here, we present the Graph Positional and Structural Encoder (GPSE), the first-ever graph encoder designed to capture rich PSE representations for augmenting any GNN. GPSE learns an efficient common latent representation for multiple PSEs, and is highly transferable: The encoder trained on a particular graph dataset can be used effectively on datasets drawn from markedly different distributions and modalities. We show that across a wide range of benchmarks, GPSE-enhanced models can significantly outperform those that employ explicitly computed PSEs, and at least match their performance in others. Our results pave the way for the development of foundational pre-trained graph encoders for extracting positional and structural information, and highlight their potential as a more powerful and efficient alternative to explicitly computed PSEs and existing self-supervised pre-training approaches. Our framework and pre-trained models are publicly available at https://github.com/G-Taxonomy-Workgroup/GPSE. For convenience, GPSE has also been integrated into the PyG library to facilitate downstream applications.
Paper Structure (41 sections, 1 theorem, 24 equations, 7 figures, 24 tables)

This paper contains 41 sections, 1 theorem, 24 equations, 7 figures, 24 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Methods
  4. Self-supervision via positional and structural encodings (PSE)
  5. GPSE architecture
  6. Training GPSE
  7. Experiments
  8. GPSE successfully predicts a wide range of target PSEs
  9. Enhancing performance on molecular graph data
  10. Transferability across diverse graph benchmarks
  11. Expressiveness of GPSE encodings
  12. Efficiency & scaling of GPSE
  13. Ablation studies
  14. GPSE makes good use of depth and global message passing from the VN
  15. Conclusion
  16. ...and 26 more sections

Key Result

Proposition 3.4

The balanced Forman Curvature is increased for most edges when adding a virtual node such that where $d_i$ is the degree of the most connected node of the edge $(i,j)$ and $\delta = d_i-d_j$.

Figures (7)

  • Figure 1: Overview of Graph Positional and Structural Encoder (GPSE) training and application.
  • Figure 2: Virtual node (VN), convolution type, and layers ablation using 5% MolPCBA for training GPSE. The y-axis denotes the GPSE average test $R^2$ score over all six PSEs, as per Table \ref{['tab:pse_perf']}.
  • Figure 5.1: 2D PCA visualization of GPSE encodings on 1-WL indistinguishable graph pairs. Applying GPSE with randomly initialized node features results in distinct encodings for HEXAGON (indigo) and PENTAGON (orange) graphs (left). The same graphs cannot be distinguished by our encoder when the node features are set to 1 for each node (right).
  • Figure 7.1: Scaling of PSE computation time with respect to number of graphs as % of MolPCBA dataset used. Visualization of Table \ref{['tab:graph_number_scaling']}.
  • Figure 7.2: Scaling experiments with respect to size of graphs, keeping the number of graphs in each dataset constant. Visualization of Table \ref{['tab:graph_size_scaling']}.
  • ...and 2 more figures

Theorems & Definitions (5)

  • Definition 3.1: Over-squashing
  • Definition 3.2: Over-smoothing
  • Definition 3.3: Graph curvature
  • Proposition 3.4
  • Definition 5.1: Graph isomorphism