Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Neural Networks with Diverse Spectral Filtering

Jingwei Guo, Kaizhu Huang, Xinping Yi, Rui Zhang

TL;DR

This paper tackles the limitation of homogeneous spectral filtering in spectral Graph Neural Networks when dealing with graphs that exhibit regional heterogeneity. It proposes Diverse Spectral Filtering (DSF), a framework that learns node-specific filter weights by combining a global shared component with a local, position-aware adjustment, and it is plug-and-play for existing backbones such as GPR-GNN, BernNet, and JacobiConv. DSF also introduces Iterative Positional Encoding (IPE) to encode node positions and Local-Global Weight Decomposition (LGWD) to separate global structure from local variation, improving interpretability. Empirical results across 11 real-world datasets show consistent gains (up to 4.92% on heterophilic graphs) and demonstrate that the learned diverse filters reflect both global characteristics and regional heterogeneity, enabling better micro-graph mining and interpretability.

Abstract

Spectral Graph Neural Networks (GNNs) have achieved tremendous success in graph machine learning, with polynomial filters applied for graph convolutions, where all nodes share the identical filter weights to mine their local contexts. Despite the success, existing spectral GNNs usually fail to deal with complex networks (e.g., WWW) due to such homogeneous spectral filtering setting that ignores the regional heterogeneity as typically seen in real-world networks. To tackle this issue, we propose a novel diverse spectral filtering (DSF) framework, which automatically learns node-specific filter weights to exploit the varying local structure properly. Particularly, the diverse filter weights consist of two components -- A global one shared among all nodes, and a local one that varies along network edges to reflect node difference arising from distinct graph parts -- to balance between local and global information. As such, not only can the global graph characteristics be captured, but also the diverse local patterns can be mined with awareness of different node positions. Interestingly, we formulate a novel optimization problem to assist in learning diverse filters, which also enables us to enhance any spectral GNNs with our DSF framework. We showcase the proposed framework on three state-of-the-arts including GPR-GNN, BernNet, and JacobiConv. Extensive experiments over 10 benchmark datasets demonstrate that our framework can consistently boost model performance by up to 4.92% in node classification tasks, producing diverse filters with enhanced interpretability. Code is available at \url{https://github.com/jingweio/DSF}.

Graph Neural Networks with Diverse Spectral Filtering

TL;DR

This paper tackles the limitation of homogeneous spectral filtering in spectral Graph Neural Networks when dealing with graphs that exhibit regional heterogeneity. It proposes Diverse Spectral Filtering (DSF), a framework that learns node-specific filter weights by combining a global shared component with a local, position-aware adjustment, and it is plug-and-play for existing backbones such as GPR-GNN, BernNet, and JacobiConv. DSF also introduces Iterative Positional Encoding (IPE) to encode node positions and Local-Global Weight Decomposition (LGWD) to separate global structure from local variation, improving interpretability. Empirical results across 11 real-world datasets show consistent gains (up to 4.92% on heterophilic graphs) and demonstrate that the learned diverse filters reflect both global characteristics and regional heterogeneity, enabling better micro-graph mining and interpretability.

Abstract

Spectral Graph Neural Networks (GNNs) have achieved tremendous success in graph machine learning, with polynomial filters applied for graph convolutions, where all nodes share the identical filter weights to mine their local contexts. Despite the success, existing spectral GNNs usually fail to deal with complex networks (e.g., WWW) due to such homogeneous spectral filtering setting that ignores the regional heterogeneity as typically seen in real-world networks. To tackle this issue, we propose a novel diverse spectral filtering (DSF) framework, which automatically learns node-specific filter weights to exploit the varying local structure properly. Particularly, the diverse filter weights consist of two components -- A global one shared among all nodes, and a local one that varies along network edges to reflect node difference arising from distinct graph parts -- to balance between local and global information. As such, not only can the global graph characteristics be captured, but also the diverse local patterns can be mined with awareness of different node positions. Interestingly, we formulate a novel optimization problem to assist in learning diverse filters, which also enables us to enhance any spectral GNNs with our DSF framework. We showcase the proposed framework on three state-of-the-arts including GPR-GNN, BernNet, and JacobiConv. Extensive experiments over 10 benchmark datasets demonstrate that our framework can consistently boost model performance by up to 4.92% in node classification tasks, producing diverse filters with enhanced interpretability. Code is available at \url{https://github.com/jingweio/DSF}.
Paper Structure (31 sections, 1 theorem, 12 equations, 7 figures, 5 tables, 1 algorithm)

This paper contains 31 sections, 1 theorem, 12 equations, 7 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Spectral Graph Neural Networks
  4. Graphs with Complex Linking Patterns
  5. Positional Encoding
  6. Notations and Preliminaries
  7. Laplacian Decomposition
  8. Graph Spectral Filtering
  9. diverse spectral filtering
  10. Motivations
  11. Diverse Filtering Framework
  12. Position-aware Filter Weights
  13. Local and Global Weight Decomposition
  14. Overall Algorithm
  15. Model Analysis
  16. ...and 16 more sections

Key Result

Proposition 1

Suppose a K-order polynomial function $f: [0, 2] \to \mathbb{R}$ with polynomial basis $P_k(\cdot)$ and coefficients $\{\alpha_k\}_{k=0}^K$ in real number. For any pair of variables $x,\hat{x} \in [0,2]$ satisfying $x = \xi \hat{x}$ where $\xi$ is a constant real number, we always have a function $g

Figures (7)

  • Figure 1: (a)-(c) Diverse filters learned from real-world networks, where five representative curves are plotted for illustration. On each graph, these filters display similar overall shapes but different local details in function curves, showing the capability of our DSF in capturing both the global graph structure and locally varied linking patterns. (d) Visualization of node-specific filter weights on Cornell dataset, where alike color indicates similar filter weights between nodes. Overall, nodes can be differentiated based on their disjoint underlying regions as circled by the blue and green dashed lines, and far-reaching nodes can still learn similar filter weights due to their akin local structures. E.g., vertices on the graph border are mostly ingrained in a line subgraph such as $\bullet-\bullet-\bullet$, and some unusual cases can be handled (see details in Section \ref{['sec:intep_filter_weight']}). These results justify the enhanced model interpretability by learning diverse spectral filters on the micro level.
  • Figure 2: Distributions of two graph properties on various real graph data (see details in Section \ref{['sec:motivation']}).
  • Figure 3: Ablation study of DSF framework on six datasets with our variants DSF-$x$-R for all $x \in \{\text{GPR}, \text{Bern}, \text{Jacobi}\}$ as an example.
  • Figure 4: Diverse filters on homophilic graphs, which are learned to be similar due to the intrinsic assortative linking patterns distributed uniformly on these networks. Our DSF presents one general framework which can be adaptive to different types of networks.
  • Figure 5: Visualization of node-specific filter weights on Chameleon dataset, where a few border nodes are cropped away for better picturing. We annotate the captured regional distinctions using irregular circles with different colors.
  • ...and 2 more figures

Theorems & Definitions (3)

  • Definition 1: Local Label Homophily
  • Definition 2: Local Graph Frequency
  • Proposition 1