Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on The Expressive Power of Graph Neural Networks

Ryoma Sato

TL;DR

This survey analyzes the expressive power of graph neural networks (GNNs), focusing on theoretical limits and provably powerful variants. It connects GNN expressivity to the Weisfeiler-Lehman (WL) graph isomorphism tests, showing vanilla message-passing GNNs are bounded by 1-WL, while models like GIN and higher-order GNNs reach higher WL powers. The work surveys higher-order, relational pooling, and randomized feature approaches to overcome GNN limitations, and it links these capabilities to distributed local algorithms and complexity considerations through the XS correspondence. It also discusses practical trade-offs in memory and time, and highlights how randomized features can dramatically improve substructure counting and approximation performance. Overall, the paper provides a comprehensive framework for understanding when and how GNNs can distinguish graph structures and solve combinatorial problems.

Abstract

Graph neural networks (GNNs) are effective machine learning models for various graph learning problems. Despite their empirical successes, the theoretical limitations of GNNs have been revealed recently. Consequently, many GNN models have been proposed to overcome these limitations. In this survey, we provide a comprehensive overview of the expressive power of GNNs and provably powerful variants of GNNs.

A Survey on The Expressive Power of Graph Neural Networks

TL;DR

This survey analyzes the expressive power of graph neural networks (GNNs), focusing on theoretical limits and provably powerful variants. It connects GNN expressivity to the Weisfeiler-Lehman (WL) graph isomorphism tests, showing vanilla message-passing GNNs are bounded by 1-WL, while models like GIN and higher-order GNNs reach higher WL powers. The work surveys higher-order, relational pooling, and randomized feature approaches to overcome GNN limitations, and it links these capabilities to distributed local algorithms and complexity considerations through the XS correspondence. It also discusses practical trade-offs in memory and time, and highlights how randomized features can dramatically improve substructure counting and approximation performance. Overall, the paper provides a comprehensive framework for understanding when and how GNNs can distinguish graph structures and solve combinatorial problems.

Abstract

Graph neural networks (GNNs) are effective machine learning models for various graph learning problems. Despite their empirical successes, the theoretical limitations of GNNs have been revealed recently. Consequently, many GNN models have been proposed to overcome these limitations. In this survey, we provide a comprehensive overview of the expressive power of GNNs and provably powerful variants of GNNs.

Paper Structure

This paper contains 17 sections, 27 theorems, 13 equations, 13 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Notations
  3. Problem Setting
  4. Graph Neural Networks
  5. Graphs That GNNs Cannot Distinguish
  6. Connection with The WL Algorithm
  7. Weisfeiler-Lehman Algorithm
  8. Connection between GNNs And The WL Algorithm
  9. Higher Order Graph Neural Networks
  10. Relational Pooling
  11. Connection with Combinatorial Problems
  12. Distributed Local Algorithms
  13. Connection with Local Algorithms
  14. Random Features Strengthen GNNs
  15. Time Complexity
  16. ...and 2 more sections

Key Result

Theorem 1

For any message passing GNN and for any graphs $G$ and $H$, if the $1$-WL algorithm outputs that $G$ and $H$ are "possibly isomorphic", the embeddings ${\boldsymbol{h}}_G$ and ${\boldsymbol{h}}_H$ computed by the GNN are the same.

Figures (13)

  • Figure 1: One-layered message passing graph neural networks.
  • Figure 2: Two-layered message passing graph neural networks.
  • Figure 3: GNNs cannot distinguish these two molecules because both are $3$-regular graphs with $20$ nodes.
  • Figure 4: Message passing GNNs cannot distinguish any pair of regular graphs with the same degree and size even if they are not isomorphic.
  • Figure 5: Although these graphs are not isomorphic or regular, GNNs cannot distinguish (a) from (b), (c) from (d), and (e) from (f)
  • ...and 8 more figures

Theorems & Definitions (30)

  • Theorem 1: GINkGNN
  • Theorem 2: GIN
  • Corollary 3
  • Theorem 4: kGNN, Proposition 4
  • Definition 5: Invariance
  • Definition 6: Equivariance
  • Theorem 7: maron2019invariant, Proposition 2
  • Theorem 8: maron2019invariant
  • Theorem 9: maron2019provably, Theorem 1
  • Theorem 10: chen2020can, Theorem 6
  • ...and 20 more