Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Spectral Graph Neural Networks with Spatially Adaptive Filtering

Jingwei Guo, Kaizhu Huang, Xinping Yi, Zixian Su, Rui Zhang

TL;DR

This paper establishes a theoretical connection between spectral filtering and spatial aggregation and proposes a novel Spatially Adaptive Filtering (SAF) framework, which leverages the adapted new graph by spectral filtering for an auxiliary non-local aggregation.

Abstract

Whilst spectral Graph Neural Networks (GNNs) are theoretically well-founded in the spectral domain, their practical reliance on polynomial approximation implies a profound linkage to the spatial domain. As previous studies rarely examine spectral GNNs from the spatial perspective, their spatial-domain interpretability remains elusive, e.g., what information is essentially encoded by spectral GNNs in the spatial domain? In this paper, to answer this question, we establish a theoretical connection between spectral filtering and spatial aggregation, unveiling an intrinsic interaction that spectral filtering implicitly leads the original graph to an adapted new graph, explicitly computed for spatial aggregation. Both theoretical and empirical investigations reveal that the adapted new graph not only exhibits non-locality but also accommodates signed edge weights to reflect label consistency among nodes. These findings thus highlight the interpretable role of spectral GNNs in the spatial domain and inspire us to rethink graph spectral filters beyond the fixed-order polynomials, which neglect global information. Built upon the theoretical findings, we revisit the state-of-the-art spectral GNNs and propose a novel Spatially Adaptive Filtering (SAF) framework, which leverages the adapted new graph by spectral filtering for an auxiliary non-local aggregation. Notably, our proposed SAF comprehensively models both node similarity and dissimilarity from a global perspective, therefore alleviating persistent deficiencies of GNNs related to long-range dependencies and graph heterophily. Extensive experiments over 13 node classification benchmarks demonstrate the superiority of our proposed framework to the state-of-the-art models.

Rethinking Spectral Graph Neural Networks with Spatially Adaptive Filtering

TL;DR

This paper establishes a theoretical connection between spectral filtering and spatial aggregation and proposes a novel Spatially Adaptive Filtering (SAF) framework, which leverages the adapted new graph by spectral filtering for an auxiliary non-local aggregation.

Abstract

Whilst spectral Graph Neural Networks (GNNs) are theoretically well-founded in the spectral domain, their practical reliance on polynomial approximation implies a profound linkage to the spatial domain. As previous studies rarely examine spectral GNNs from the spatial perspective, their spatial-domain interpretability remains elusive, e.g., what information is essentially encoded by spectral GNNs in the spatial domain? In this paper, to answer this question, we establish a theoretical connection between spectral filtering and spatial aggregation, unveiling an intrinsic interaction that spectral filtering implicitly leads the original graph to an adapted new graph, explicitly computed for spatial aggregation. Both theoretical and empirical investigations reveal that the adapted new graph not only exhibits non-locality but also accommodates signed edge weights to reflect label consistency among nodes. These findings thus highlight the interpretable role of spectral GNNs in the spatial domain and inspire us to rethink graph spectral filters beyond the fixed-order polynomials, which neglect global information. Built upon the theoretical findings, we revisit the state-of-the-art spectral GNNs and propose a novel Spatially Adaptive Filtering (SAF) framework, which leverages the adapted new graph by spectral filtering for an auxiliary non-local aggregation. Notably, our proposed SAF comprehensively models both node similarity and dissimilarity from a global perspective, therefore alleviating persistent deficiencies of GNNs related to long-range dependencies and graph heterophily. Extensive experiments over 13 node classification benchmarks demonstrate the superiority of our proposed framework to the state-of-the-art models.
Paper Structure (43 sections, 4 theorems, 7 equations, 7 figures, 7 tables)

This paper contains 43 sections, 4 theorems, 7 equations, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Motivation and Related Works
  3. Graph Neural Networks
  4. Unified Viewpoints for GNNs.
  5. Long-range Dependencies
  6. Graph Heterophily
  7. Notations and Preliminaries
  8. Spectral Filtering
  9. Spatial Aggregation
  10. Rethinking Spectral Graph Neural Networks from the Spatial Perspective
  11. Interplay of Spectral and Spatial Domains through the Lens of Graph Optimization
  12. Closed-form Solution
  13. Iterative Solution
  14. Theoretical Interaction --- the Adapted New Graph
  15. What are the differences between $\hat{\mathbf{A}}^\text{new}$ and $g_\psi(\hat{\mathbf{L}})$
  16. ...and 28 more sections

Key Result

Lemma 1

Let $\mathbf{M} \in \mathbb{R}^{N \times N}$ be a matrix with eigenvalues $\lambda_n$, if $|\lambda_n| < 1$ for all $n=1,2, ..., N$, then $(\mathbf{I} - \mathbf{M})^{-1}$ exists and can be expanded as an infinite series, i.e., $(\mathbf{I} - \mathbf{M})^{-1} = \sum_{t=0}^\infty \mathbf{M}^t$, which

Figures (7)

  • Figure 1: Distributions of connected nodes in the new graph based on their geodesic/shortest-path distance (as $\Delta_{i,j}$) in the original graph. Nodes, distant in the original graph ($\Delta_{i,j} > 1$ in x-axis), can be linked in the new graph (Number $> 0$ in y-axis).
  • Figure 2: Left y-axis: Homophily comparison between original and new graphs, considering only positive edges (blue and yellow bars). Right y-axis: Percentage of negative edges in the new graph that connect nodes from different classes (green bar).
  • Figure 3: Illustration of SAF framework where varying node colors represent different node labels. SAF leverages the adapted new graph by spectral filtering for auxiliary non-local aggregation in the spatial domain and allows individual nodes to flexibly balance between spectral and spatial features.
  • Figure 4: Attention changing trends w.r.t. training epochs.
  • Figure 5: Ablation study of SAF framework on five datasets.
  • ...and 2 more figures

Theorems & Definitions (7)

  • Lemma 1
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof