Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PANDA: Expanded Width-Aware Message Passing Beyond Rewiring

Jeongwhan Choi, Sumin Park, Hyowon Wi, Sung-Bae Cho, Noseong Park

TL;DR

This work tackles over-squashing in graph neural networks by proposing PANDA, a width-aware message passing framework that selectively expands the hidden width of high-centrality nodes. By introducing four edge-type aggregators that enable information exchange between nodes of different widths, PANDA avoids graph rewiring while enhancing long-range propagation. Grounded in a sensitivity bound and effective resistance analysis, PANDA demonstrates superior performance to rewiring baselines on graph classification, node classification, and long-range benchmarks, while preserving original topology. The approach offers a scalable, topology-preserving alternative to rewiring, with implications for more robust and expressive GNNs in domains where domain knowledge constrains topology changes.

Abstract

Recent research in the field of graph neural network (GNN) has identified a critical issue known as "over-squashing," resulting from the bottleneck phenomenon in graph structures, which impedes the propagation of long-range information. Prior works have proposed a variety of graph rewiring concepts that aim at optimizing the spatial or spectral properties of graphs to promote the signal propagation. However, such approaches inevitably deteriorate the original graph topology, which may lead to a distortion of information flow. To address this, we introduce an expanded width-aware (PANDA) message passing, a new message passing paradigm where nodes with high centrality, a potential source of over-squashing, are selectively expanded in width to encapsulate the growing influx of signals from distant nodes. Experimental results show that our method outperforms existing rewiring methods, suggesting that selectively expanding the hidden state of nodes can be a compelling alternative to graph rewiring for addressing the over-squashing.

PANDA: Expanded Width-Aware Message Passing Beyond Rewiring

TL;DR

This work tackles over-squashing in graph neural networks by proposing PANDA, a width-aware message passing framework that selectively expands the hidden width of high-centrality nodes. By introducing four edge-type aggregators that enable information exchange between nodes of different widths, PANDA avoids graph rewiring while enhancing long-range propagation. Grounded in a sensitivity bound and effective resistance analysis, PANDA demonstrates superior performance to rewiring baselines on graph classification, node classification, and long-range benchmarks, while preserving original topology. The approach offers a scalable, topology-preserving alternative to rewiring, with implications for more robust and expressive GNNs in domains where domain knowledge constrains topology changes.

Abstract

Recent research in the field of graph neural network (GNN) has identified a critical issue known as "over-squashing," resulting from the bottleneck phenomenon in graph structures, which impedes the propagation of long-range information. Prior works have proposed a variety of graph rewiring concepts that aim at optimizing the spatial or spectral properties of graphs to promote the signal propagation. However, such approaches inevitably deteriorate the original graph topology, which may lead to a distortion of information flow. To address this, we introduce an expanded width-aware (PANDA) message passing, a new message passing paradigm where nodes with high centrality, a potential source of over-squashing, are selectively expanded in width to encapsulate the growing influx of signals from distant nodes. Experimental results show that our method outperforms existing rewiring methods, suggesting that selectively expanding the hidden state of nodes can be a compelling alternative to graph rewiring for addressing the over-squashing.
Paper Structure (78 sections, 1 theorem, 17 equations, 14 figures, 14 tables)

This paper contains 78 sections, 1 theorem, 17 equations, 14 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Motivation.
  3. Main idea.
  4. Contributions and outline.
  5. Preliminaries
  6. Message Passing Neural Networks
  7. Over-squashing Problem
  8. Sensitivity bound.
  9. Effective resistance and signal propagation.
  10. Over-smoothing problem.
  11. Proposed Method
  12. Motivation and Design Rationale
  13. Sensitivity bound.
  14. Design rationale for selecting node centrality.
  15. Expanded Width-Aware Message Passing
  16. ...and 63 more sections

Key Result

Proposition 2.1

Given the sensitivity bound in Eq. eq:sens, we consider two scenarios: one with a standard width $p$ and the other with an increased width $p_{\mathrm{high}}$, where $p_{\mathrm{high}} > p$. For nodes with increased width, the sensitivity bound is augmented, demonstrating a potentially higher sensit This increased sensitivity can potentially reduce over-squashing, facilitating better signal propag

Figures (14)

  • Figure 1: Potential pitfalls of rewiring in domain-specific graphs: (a) In a molecular graph, rewiring the edge in red to a benzene ring violates the domain knowledge. (b) In a social graph, connecting a user to his/her enemy may lead to totally different meaning.
  • Figure 2: Comparison of resistance correlation and mean accuracy across different methods for GCN. A large negative correlation reflects that a higher total effective resistance is associated with reduced signal propagation (see \ref{['sec:why']}).
  • Figure 3: Examples of PANDA's message passing mechanism. The size of the node indicates the size of hidden dimension.
  • Figure 4: Our proposed PANDA message passing framework. First, we selectively expand widths (i.e., hidden dimension sizes) according to a centrality in $\mathcal{G}$, and our PANDA message passing enables signal propagation among nodes with different widths (low-dimensional nodes and high-dimensional nodes).
  • Figure 5: Empirical sensitivity across layers for GCN on Mutag. More results on other datasets are in \ref{['app:vis-why']}.
  • ...and 9 more figures

Theorems & Definitions (1)

  • Proposition 2.1: Informal