Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding over-squashing and bottlenecks on graphs via curvature

Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, Michael M. Bronstein

TL;DR

The paper tackles over-squashing in graph neural networks by linking information bottlenecks to local graph geometry. It introduces Balanced Forman curvature, an edge-centric measure that correlates negative curvature with bottlenecks, and proves bounds connecting curvature to Jacobian-based propagation limits. To alleviate bottlenecks, it proposes the Stochastic Discrete Ricci Flow (SDRF), a curvature-guided, surgery-like graph rewiring method that preserves topology better than diffusion-based approaches. Through experiments on nine datasets with varying homophily, SDRF demonstrates robust improvements, especially in low-homophily settings, and is shown to meaningfully reduce bottleneck effects while maintaining graph structure. The work provides a principled, geometry-based alternative to diffusion rewiring for enhancing long-range information flow in GNNs.

Abstract

Most graph neural networks (GNNs) use the message passing paradigm, in which node features are propagated on the input graph. Recent works pointed to the distortion of information flowing from distant nodes as a factor limiting the efficiency of message passing for tasks relying on long-distance interactions. This phenomenon, referred to as 'over-squashing', has been heuristically attributed to graph bottlenecks where the number of $k$-hop neighbors grows rapidly with $k$. We provide a precise description of the over-squashing phenomenon in GNNs and analyze how it arises from bottlenecks in the graph. For this purpose, we introduce a new edge-based combinatorial curvature and prove that negatively curved edges are responsible for the over-squashing issue. We also propose and experimentally test a curvature-based graph rewiring method to alleviate the over-squashing.

Understanding over-squashing and bottlenecks on graphs via curvature

TL;DR

The paper tackles over-squashing in graph neural networks by linking information bottlenecks to local graph geometry. It introduces Balanced Forman curvature, an edge-centric measure that correlates negative curvature with bottlenecks, and proves bounds connecting curvature to Jacobian-based propagation limits. To alleviate bottlenecks, it proposes the Stochastic Discrete Ricci Flow (SDRF), a curvature-guided, surgery-like graph rewiring method that preserves topology better than diffusion-based approaches. Through experiments on nine datasets with varying homophily, SDRF demonstrates robust improvements, especially in low-homophily settings, and is shown to meaningfully reduce bottleneck effects while maintaining graph structure. The work provides a principled, geometry-based alternative to diffusion rewiring for enhancing long-range information flow in GNNs.

Abstract

Most graph neural networks (GNNs) use the message passing paradigm, in which node features are propagated on the input graph. Recent works pointed to the distortion of information flowing from distant nodes as a factor limiting the efficiency of message passing for tasks relying on long-distance interactions. This phenomenon, referred to as 'over-squashing', has been heuristically attributed to graph bottlenecks where the number of -hop neighbors grows rapidly with . We provide a precise description of the over-squashing phenomenon in GNNs and analyze how it arises from bottlenecks in the graph. For this purpose, we introduce a new edge-based combinatorial curvature and prove that negatively curved edges are responsible for the over-squashing issue. We also propose and experimentally test a curvature-based graph rewiring method to alleviate the over-squashing.
Paper Structure (38 sections, 18 theorems, 85 equations, 6 figures, 10 tables, 1 algorithm)

This paper contains 38 sections, 18 theorems, 85 equations, 6 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions and Outline.
  3. Analysis of the over-squashing phenomenon
  4. Preliminaries
  5. Message passing neural networks (MPNNs).
  6. The over-squashing problem
  7. Sensitivity analysis.
  8. Graph curvature and bottleneck
  9. Curvatures on graphs.
  10. Balanced Forman curvature.
  11. Curvature and over-squashing.
  12. Bottleneck via Cheeger constant.
  13. Curvature-based rewiring methods
  14. Curvature-based graph rewiring.
  15. Graph structure preservation.
  16. ...and 23 more sections

Key Result

Lemma 0

Assume an MPNN as in equation eq:MPNN. Let $i,s\in V$ with $s\in S_{r+1}(i)$. If $\lvert \nabla \phi_{\ell}\rvert \leq \alpha$ and $\lvert \nabla \psi_{\ell}\rvert \leq \beta$ for $0 \leq \ell \leq r$, then

Figures (6)

  • Figure 1: Top: evolution of curvature on a surface may reduce the bottleneck. Bottom: this paper shows how the same may be done on graphs to improve GNN performance. Blue/red shows negative/positive curvature.
  • Figure 2: Different regimes of curvatures on graphs analogous to spherical (a), planar (b), and hyperbolic (c) geometries in the continuous setting.
  • Figure 3: 4-cycle contribution.
  • Figure 4: Comparing the degree distribution of the original graphs to the preprocessed version. The x-axis is node degree in $\text{log}_2$ scale, and the plots are a kernel density estimate of the degree distribution. In the captions we see the Wasserstein distance $W_1$ between the original and preprocessed graphs.
  • Figure 5: Comparing the degree distribution of the original graphs to the preprocessed version. The x-axis is node degree in $\text{log}_2$ scale, and the plots are a kernel density estimate of the degree distribution. In the captions we see the Wasserstein distance $W_1$ between the original and preprocessed graphs.
  • ...and 1 more figures

Theorems & Definitions (44)

  • Lemma 0
  • Definition 1: Balanced Forman curvature
  • Theorem 1
  • Corollary 1
  • Theorem 2
  • Proposition 2
  • Theorem 3
  • Lemma 3
  • proof
  • Corollary 3
  • ...and 34 more