Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Over-Squashing in Graph Neural Networks: A Comprehensive survey

Singh Akansha

TL;DR

This survey addresses the challenge of over-squashing in graph neural networks, where information from distant nodes is bottlenecked during long-range message passing. It surveys a broad landscape of mitigation strategies, organized into spatial and spectral graph rewiring, curvature-based methods, and hybrid approaches, as well as graph-transformer paradigms that bypass some locality constraints. A systematic taxonomy is provided alongside discussions of trade-offs with over-smoothing, and the work catalogs common node- and graph-level benchmarks to evaluate these methods. The analysis highlights both theoretical insights and practical considerations, including scalability, applicability to dynamic graphs, and robustness, offering a roadmap for designing GNNs that preserve long-range information without eroding locality. The paper thus serves as a comprehensive reference for researchers and practitioners aiming to mitigate over-squashing in real-world graph learning tasks.

Abstract

Graph Neural Networks (GNNs) revolutionize machine learning for graph-structured data, effectively capturing complex relationships. They disseminate information through interconnected nodes, but long-range interactions face challenges known as "over-squashing". This survey delves into the challenge of over-squashing in Graph Neural Networks (GNNs), where long-range information dissemination is hindered, impacting tasks reliant on intricate long-distance interactions. It comprehensively explores the causes, consequences, and mitigation strategies for over-squashing. Various methodologies are reviewed, including graph rewiring, novel normalization, spectral analysis, and curvature-based strategies, with a focus on their trade-offs and effectiveness. The survey also discusses the interplay between over-squashing and other GNN limitations, such as over-smoothing, and provides a taxonomy of models designed to address these issues in node and graph-level tasks. Benchmark datasets for performance evaluation are also detailed, making this survey a valuable resource for researchers and practitioners in the GNN field.

Over-Squashing in Graph Neural Networks: A Comprehensive survey

TL;DR

This survey addresses the challenge of over-squashing in graph neural networks, where information from distant nodes is bottlenecked during long-range message passing. It surveys a broad landscape of mitigation strategies, organized into spatial and spectral graph rewiring, curvature-based methods, and hybrid approaches, as well as graph-transformer paradigms that bypass some locality constraints. A systematic taxonomy is provided alongside discussions of trade-offs with over-smoothing, and the work catalogs common node- and graph-level benchmarks to evaluate these methods. The analysis highlights both theoretical insights and practical considerations, including scalability, applicability to dynamic graphs, and robustness, offering a roadmap for designing GNNs that preserve long-range information without eroding locality. The paper thus serves as a comprehensive reference for researchers and practitioners aiming to mitigate over-squashing in real-world graph learning tasks.

Abstract

Graph Neural Networks (GNNs) revolutionize machine learning for graph-structured data, effectively capturing complex relationships. They disseminate information through interconnected nodes, but long-range interactions face challenges known as "over-squashing". This survey delves into the challenge of over-squashing in Graph Neural Networks (GNNs), where long-range information dissemination is hindered, impacting tasks reliant on intricate long-distance interactions. It comprehensively explores the causes, consequences, and mitigation strategies for over-squashing. Various methodologies are reviewed, including graph rewiring, novel normalization, spectral analysis, and curvature-based strategies, with a focus on their trade-offs and effectiveness. The survey also discusses the interplay between over-squashing and other GNN limitations, such as over-smoothing, and provides a taxonomy of models designed to address these issues in node and graph-level tasks. Benchmark datasets for performance evaluation are also detailed, making this survey a valuable resource for researchers and practitioners in the GNN field.
Paper Structure (11 sections, 7 theorems, 37 equations, 5 tables)

This paper contains 11 sections, 7 theorems, 37 equations, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Graph Neural Networks
  4. Over-squashing
  5. Handling over-squashing in GNNs
  6. Spatial Graph Rewiring Methods:
  7. Spectral Graph Rewiring Methods:
  8. Unifying Approaches to Address Over-Squashing and Over-Smoothing Trade-offs in Graph Neural Networks
  9. Graph Transformers and other Strategies
  10. Datasets
  11. Conclusion and Future Direction

Key Result

Theorem 2.1

For a connected graph $G$, let $u, v \in V$ and $\| \nabla UP_l \| \leq \alpha$ and $\max \{ \| \nabla AGG_l \|, 1 \} \leq \beta$ for all $l = 0, \ldots, r$. Let $d_{\text{max}}$ and $d_{\text{min}}$ be the maximum and minimum degrees of nodes $u$ and $v$, and $\max \{|\mu_2|, |\mu_n|\} \leq \mu$ wh

Theorems & Definitions (15)

  • Definition 2.1
  • Theorem 2.1
  • Theorem 2.2
  • Definition 2.2
  • Definition 2.3
  • Definition 3.1
  • Theorem 3.1
  • Definition 3.2
  • Theorem 3.2
  • Theorem 4.1
  • ...and 5 more