Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Return of ChebNet: Understanding and Improving an Overlooked GNN on Long Range Tasks

Ali Hariri, Álvaro Arroyo, Alessio Gravina, Moshe Eliasof, Carola-Bibiane Schönlieb, Davide Bacciu, Kamyar Azizzadenesheli, Xiaowen Dong, Pierre Vandergheynst

TL;DR

ChebNet is revisited to address long-range dependencies, revealing that $K$-order Chebyshev filters can be competitive but risk unstable dynamics as the receptive field grows. The authors formulate Stable-ChebNet by enforcing antisymmetric weights in an ODE framework and solving with forward-Euler discretization, yielding non-dissipative propagation with purely imaginary Jacobian spectra and avoiding eigendecompositions or rewiring. Theoretical guarantees accompany empirical validation across OGB, LRGB, and heterophilic benchmarks, where Stable-ChebNet attains near state-of-the-art performance while preserving Chebyshev filter structure. Overall, the work repositions spectral GNNs as scalable, principled tools for long-range graph modeling with robust stability properties.

Abstract

ChebNet, one of the earliest spectral GNNs, has largely been overshadowed by Message Passing Neural Networks (MPNNs), which gained popularity for their simplicity and effectiveness in capturing local graph structure. Despite their success, MPNNs are limited in their ability to capture long-range dependencies between nodes. This has led researchers to adapt MPNNs through rewiring or make use of Graph Transformers, which compromises the computational efficiency that characterized early spatial message-passing architectures, and typically disregards the graph structure. Almost a decade after its original introduction, we revisit ChebNet to shed light on its ability to model distant node interactions. We find that out-of-box, ChebNet already shows competitive advantages relative to classical MPNNs and GTs on long-range benchmarks, while maintaining good scalability properties for high-order polynomials. However, we uncover that this polynomial expansion leads ChebNet to an unstable regime during training. To address this limitation, we cast ChebNet as a stable and non-dissipative dynamical system, which we coin Stable-ChebNet. Our Stable-ChebNet model allows for stable information propagation, and has controllable dynamics which do not require the use of eigendecompositions, positional encodings, or graph rewiring. Across several benchmarks, Stable-ChebNet achieves near state-of-the-art performance.

Return of ChebNet: Understanding and Improving an Overlooked GNN on Long Range Tasks

TL;DR

ChebNet is revisited to address long-range dependencies, revealing that -order Chebyshev filters can be competitive but risk unstable dynamics as the receptive field grows. The authors formulate Stable-ChebNet by enforcing antisymmetric weights in an ODE framework and solving with forward-Euler discretization, yielding non-dissipative propagation with purely imaginary Jacobian spectra and avoiding eigendecompositions or rewiring. Theoretical guarantees accompany empirical validation across OGB, LRGB, and heterophilic benchmarks, where Stable-ChebNet attains near state-of-the-art performance while preserving Chebyshev filter structure. Overall, the work repositions spectral GNNs as scalable, principled tools for long-range graph modeling with robust stability properties.

Abstract

ChebNet, one of the earliest spectral GNNs, has largely been overshadowed by Message Passing Neural Networks (MPNNs), which gained popularity for their simplicity and effectiveness in capturing local graph structure. Despite their success, MPNNs are limited in their ability to capture long-range dependencies between nodes. This has led researchers to adapt MPNNs through rewiring or make use of Graph Transformers, which compromises the computational efficiency that characterized early spatial message-passing architectures, and typically disregards the graph structure. Almost a decade after its original introduction, we revisit ChebNet to shed light on its ability to model distant node interactions. We find that out-of-box, ChebNet already shows competitive advantages relative to classical MPNNs and GTs on long-range benchmarks, while maintaining good scalability properties for high-order polynomials. However, we uncover that this polynomial expansion leads ChebNet to an unstable regime during training. To address this limitation, we cast ChebNet as a stable and non-dissipative dynamical system, which we coin Stable-ChebNet. Our Stable-ChebNet model allows for stable information propagation, and has controllable dynamics which do not require the use of eigendecompositions, positional encodings, or graph rewiring. Across several benchmarks, Stable-ChebNet achieves near state-of-the-art performance.

Paper Structure

This paper contains 30 sections, 6 theorems, 32 equations, 7 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Contributions and Outline
  3. Background
  4. Background on Spectral Graph Neural Networks
  5. MPNNs and their Limitations
  6. The Connection between ChebNet and GCN
  7. Analyzing and Improving ChebNet from First Principles
  8. The Effectiveness and Scalability of Vanilla ChebNet
  9. Signal Propagation Analysis of ChebNet
  10. Stable-ChebNet: Stability with Antisymmetric Parameterization
  11. Experiments
  12. Open-Graph Benchmark.
  13. Long-Range Graph Benchmark (LRGB).
  14. Heterophilic benchmarks
  15. Conclusion
  16. ...and 15 more sections

Key Result

Lemma 3.1

Consider a linear spectral GNN whose layer-wise update is performed through the following polynomial filter $f(\mathbf{X}) = \sum_{k=1}^{K} T_k({\bf L}) \, \mathbf{X} \, \Theta_k$, where $\mathbf{X} \in \mathbb{R}^{n \times d}$ is the node feature matrix, $T_k({\bf L})\in\mathbb{R}^{n\times n}$ is t

Figures (7)

  • Figure 1: Top: Vanilla ChebNet. Bottom: Stable-ChebNet. While the original ChebNet’s high-order Chebyshev filters induce unstable dynamics resulting in dissipative behavior, our Stable-ChebNet yields bounded propagation through layers.
  • Figure 2: Epoch times for DRew and ChebNet for different receptive fields on the peptides-func task. M is the number of layers, and K is the number of filters.
  • Figure 3: Test accuracy on RingTransfer.
  • Figure 4: Singular‐value spectra of the graph‐wise Jacobian in the complex plane for vanilla ChebNet with increasing polynomial order $K$.
  • Figure 5: Mean Squared Error (MSE) comparison of various MPNN baselines at different node counts $N$ of Barbell graphs.
  • ...and 2 more figures

Theorems & Definitions (11)

  • Lemma 3.1: Layer-Wise Jacobian for a Spectral GNN
  • Theorem 1: Layer-Wise Jacobian singular-value distribution
  • Theorem 2: ChebNet Sensitivity
  • Theorem 3: Purely Imaginary Eigenvalues
  • Theorem 4: Non-exponential Information Growth or Decay with Antisymmetric Weights
  • proof
  • proof
  • proof
  • proof
  • Theorem 5: Sensitivity uppperbound of standard MPNNs, taken from di2023over
  • ...and 1 more