Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Expressive Power of Spectral Graph Neural Networks with Eigenvalue Correction

Kangkang Lu, Yanhua Yu, Hao Fei, Xuan Li, Zixuan Yang, Zirui Guo, Meiyu Liang, Mengran Yin, Tat-Seng Chua

TL;DR

This paper empirically observes that normalized Laplacian matrices frequently possess repeated eigenvalues, and theoretically establishes that the number of distinguishable eigenvalues plays a pivotal role in determining the expressive power of spectral graph neural networks.

Abstract

In recent years, spectral graph neural networks, characterized by polynomial filters, have garnered increasing attention and have achieved remarkable performance in tasks such as node classification. These models typically assume that eigenvalues for the normalized Laplacian matrix are distinct from each other, thus expecting a polynomial filter to have a high fitting ability. However, this paper empirically observes that normalized Laplacian matrices frequently possess repeated eigenvalues. Moreover, we theoretically establish that the number of distinguishable eigenvalues plays a pivotal role in determining the expressive power of spectral graph neural networks. In light of this observation, we propose an eigenvalue correction strategy that can free polynomial filters from the constraints of repeated eigenvalue inputs. Concretely, the proposed eigenvalue correction strategy enhances the uniform distribution of eigenvalues, thus mitigating repeated eigenvalues, and improving the fitting capacity and expressive power of polynomial filters. Extensive experimental results on both synthetic and real-world datasets demonstrate the superiority of our method. The code is available at: https://github.com/Lukangkang123/EC-GNN

Improving Expressive Power of Spectral Graph Neural Networks with Eigenvalue Correction

TL;DR

This paper empirically observes that normalized Laplacian matrices frequently possess repeated eigenvalues, and theoretically establishes that the number of distinguishable eigenvalues plays a pivotal role in determining the expressive power of spectral graph neural networks.

Abstract

In recent years, spectral graph neural networks, characterized by polynomial filters, have garnered increasing attention and have achieved remarkable performance in tasks such as node classification. These models typically assume that eigenvalues for the normalized Laplacian matrix are distinct from each other, thus expecting a polynomial filter to have a high fitting ability. However, this paper empirically observes that normalized Laplacian matrices frequently possess repeated eigenvalues. Moreover, we theoretically establish that the number of distinguishable eigenvalues plays a pivotal role in determining the expressive power of spectral graph neural networks. In light of this observation, we propose an eigenvalue correction strategy that can free polynomial filters from the constraints of repeated eigenvalue inputs. Concretely, the proposed eigenvalue correction strategy enhances the uniform distribution of eigenvalues, thus mitigating repeated eigenvalues, and improving the fitting capacity and expressive power of polynomial filters. Extensive experimental results on both synthetic and real-world datasets demonstrate the superiority of our method. The code is available at: https://github.com/Lukangkang123/EC-GNN
Paper Structure (27 sections, 4 theorems, 17 equations, 7 figures, 6 tables)

This paper contains 27 sections, 4 theorems, 17 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Spectral GNNs
  4. Expressive Power of Spectral GNNs
  5. Distribution of Eigenvalues of Normalized Laplacian Matrix
  6. Proposed Method
  7. Preliminaries
  8. Expressive Power of Existing Polynomial Spectral GNNs
  9. Eigenvalue Correction
  10. Combination of Corrected Eigenvalues with Existing Polynomial Filters
  11. Insertion into GPR-GNN.
  12. Insertion into BernNet.
  13. Insertion into JacobiConv.
  14. Experiment
  15. Evaluation on Learning Filters
  16. ...and 12 more sections

Key Result

Lemma 1

polynomial spectral GNNs can produce any one-dimensional prediction if $\mathbf{\hat{L}}$ has no repeated eigenvalues and $\mathbf{X}$ contains all frequency components.

Figures (7)

  • Figure 1: The distribution of eigenvalues of the normalized Laplacian matrix on different datasets. The abscissa represents eigenvalues, and the ordinate represents the probability density.
  • Figure 2: Illustration of two band-rejection filters with different numbers of eigenvalues. The red dotted line in (b) represents more eigenvalues than (a).
  • Figure 3: Histogram of the probability density distribution of the original eigenvalues and the corrected eigenvalues.
  • Figure 4: Ablation study of proposed method EC-$x$ on four datasets with our variants EC-$x$-A and EC-$x$-B for all $x \in$ {GPR, Bern, Jacobi}.
  • Figure 5: Effect of hyperparameter $\beta$ on model performance.
  • ...and 2 more figures

Theorems & Definitions (4)

  • Lemma 1
  • Theorem 1
  • Theorem 2
  • Theorem 3