Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks

Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, Martin Grohe

TL;DR

This work theoretically bridges graph neural networks (GNNs) and the Weisfeiler-Leman (WL) graph isomorphism tests, showing that standard 1-GNNs cannot surpass 1-WL in distinguishing non-isomorphic subgraphs, but can match its power with appropriate initialization. It then generalizes to k-GNNs, which operate on k-node subgraphs via the k-WL framework, providing strictly greater expressive power than traditional GNNs. To leverage real-world graph structure, the authors introduce a hierarchical variant, 1-k-GNNs, that combines representations learned at multiple granularities in an end-to-end trainable fashion, and demonstrate improved performance on graph classification and QM9 molecular regression tasks. Empirical results show that hierarchical k-GNNs often outperform 1-GNNs and are competitive with or surpass graph kernels on several benchmarks, underscoring the value of higher-order and hierarchical graph representations for complex graph tasks.

Abstract

In recent years, graph neural networks (GNNs) have emerged as a powerful neural architecture to learn vector representations of nodes and graphs in a supervised, end-to-end fashion. Up to now, GNNs have only been evaluated empirically -- showing promising results. The following work investigates GNNs from a theoretical point of view and relates them to the $1$-dimensional Weisfeiler-Leman graph isomorphism heuristic ($1$-WL). We show that GNNs have the same expressiveness as the $1$-WL in terms of distinguishing non-isomorphic (sub-)graphs. Hence, both algorithms also have the same shortcomings. Based on this, we propose a generalization of GNNs, so-called $k$-dimensional GNNs ($k$-GNNs), which can take higher-order graph structures at multiple scales into account. These higher-order structures play an essential role in the characterization of social networks and molecule graphs. Our experimental evaluation confirms our theoretical findings as well as confirms that higher-order information is useful in the task of graph classification and regression.

Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks

TL;DR

This work theoretically bridges graph neural networks (GNNs) and the Weisfeiler-Leman (WL) graph isomorphism tests, showing that standard 1-GNNs cannot surpass 1-WL in distinguishing non-isomorphic subgraphs, but can match its power with appropriate initialization. It then generalizes to k-GNNs, which operate on k-node subgraphs via the k-WL framework, providing strictly greater expressive power than traditional GNNs. To leverage real-world graph structure, the authors introduce a hierarchical variant, 1-k-GNNs, that combines representations learned at multiple granularities in an end-to-end trainable fashion, and demonstrate improved performance on graph classification and QM9 molecular regression tasks. Empirical results show that hierarchical k-GNNs often outperform 1-GNNs and are competitive with or surpass graph kernels on several benchmarks, underscoring the value of higher-order and hierarchical graph representations for complex graph tasks.

Abstract

In recent years, graph neural networks (GNNs) have emerged as a powerful neural architecture to learn vector representations of nodes and graphs in a supervised, end-to-end fashion. Up to now, GNNs have only been evaluated empirically -- showing promising results. The following work investigates GNNs from a theoretical point of view and relates them to the -dimensional Weisfeiler-Leman graph isomorphism heuristic (-WL). We show that GNNs have the same expressiveness as the -WL in terms of distinguishing non-isomorphic (sub-)graphs. Hence, both algorithms also have the same shortcomings. Based on this, we propose a generalization of GNNs, so-called -dimensional GNNs (-GNNs), which can take higher-order graph structures at multiple scales into account. These higher-order structures play an essential role in the characterization of social networks and molecule graphs. Our experimental evaluation confirms our theoretical findings as well as confirms that higher-order information is useful in the task of graph classification and regression.

Paper Structure

This paper contains 24 sections, 16 theorems, 41 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Notation and Background
  5. Weisfeiler-Leman Algorithm
  6. Graph Neural Networks
  7. Relationship Between 1-WL and 1-GNNs
  8. Shortcomings of Both Approaches
  9. $\boldsymbol{k}$-dimensional Graph Neural Networks
  10. Hierarchical Variant
  11. Experimental Study
  12. Datasets
  13. Baselines
  14. Model Configuration
  15. Experimental Protocol
  16. ...and 9 more sections

Key Result

Theorem 1

Let $(G, l)$ be a labeled graph. Then for all $t\ge 0$ and for all choices of initial colorings $f^{(0)}$ consistent with $l$, and weights $\mathbf{W}^{(t)}$,

Figures (1)

  • Figure 1: Illustration of the proposed hierarchical variant of the $k$-GNN layer. For each subgraph $S$ on $k$ nodes a feature $f$ is learned, which is initialized with the learned features of all $(k-1)$-element subgraphs of $S$. Hence, a hierarchical representation of the input graph is learned.

Theorems & Definitions (26)

  • Theorem 1
  • Theorem 2
  • Proposition 3
  • Proposition 4
  • Theorem 5: Theorem 1 in the main paper
  • proof : Proof of Theorem 1
  • Theorem 6: Theorem 2 in the main paper
  • Lemma 7
  • proof
  • Corollary 8
  • ...and 16 more