Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited

Mingguo He, Zhewei Wei, Ji-Rong Wen

TL;DR

CebNetII is proposed, a new GNN model based on Chebyshev interpolation, which enhances the original Chebyshv polynomial approximation while reducing the Runge phenomenon, and can learn arbitrary graph convolutions and achieve superior performance in both full and semi-supervised node classification tasks.

Abstract

Designing spectral convolutional networks is a challenging problem in graph learning. ChebNet, one of the early attempts, approximates the spectral graph convolutions using Chebyshev polynomials. GCN simplifies ChebNet by utilizing only the first two Chebyshev polynomials while still outperforming it on real-world datasets. GPR-GNN and BernNet demonstrate that the Monomial and Bernstein bases also outperform the Chebyshev basis in terms of learning the spectral graph convolutions. Such conclusions are counter-intuitive in the field of approximation theory, where it is established that the Chebyshev polynomial achieves the optimum convergent rate for approximating a function. In this paper, we revisit the problem of approximating the spectral graph convolutions with Chebyshev polynomials. We show that ChebNet's inferior performance is primarily due to illegal coefficients learnt by ChebNet approximating analytic filter functions, which leads to over-fitting. We then propose ChebNetII, a new GNN model based on Chebyshev interpolation, which enhances the original Chebyshev polynomial approximation while reducing the Runge phenomenon. We conducted an extensive experimental study to demonstrate that ChebNetII can learn arbitrary graph convolutions and achieve superior performance in both full- and semi-supervised node classification tasks. Most notably, we scale ChebNetII to a billion graph ogbn-papers100M, showing that spectral-based GNNs have superior performance. Our code is available at https://github.com/ivam-he/ChebNetII.

Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited

TL;DR

CebNetII is proposed, a new GNN model based on Chebyshev interpolation, which enhances the original Chebyshv polynomial approximation while reducing the Runge phenomenon, and can learn arbitrary graph convolutions and achieve superior performance in both full and semi-supervised node classification tasks.

Abstract

Designing spectral convolutional networks is a challenging problem in graph learning. ChebNet, one of the early attempts, approximates the spectral graph convolutions using Chebyshev polynomials. GCN simplifies ChebNet by utilizing only the first two Chebyshev polynomials while still outperforming it on real-world datasets. GPR-GNN and BernNet demonstrate that the Monomial and Bernstein bases also outperform the Chebyshev basis in terms of learning the spectral graph convolutions. Such conclusions are counter-intuitive in the field of approximation theory, where it is established that the Chebyshev polynomial achieves the optimum convergent rate for approximating a function. In this paper, we revisit the problem of approximating the spectral graph convolutions with Chebyshev polynomials. We show that ChebNet's inferior performance is primarily due to illegal coefficients learnt by ChebNet approximating analytic filter functions, which leads to over-fitting. We then propose ChebNetII, a new GNN model based on Chebyshev interpolation, which enhances the original Chebyshev polynomial approximation while reducing the Runge phenomenon. We conducted an extensive experimental study to demonstrate that ChebNetII can learn arbitrary graph convolutions and achieve superior performance in both full- and semi-supervised node classification tasks. Most notably, we scale ChebNetII to a billion graph ogbn-papers100M, showing that spectral-based GNNs have superior performance. Our code is available at https://github.com/ivam-he/ChebNetII.
Paper Structure (25 sections, 6 theorems, 22 equations, 6 figures, 15 tables)

This paper contains 25 sections, 6 theorems, 22 equations, 6 figures, 15 tables.

Table of Contents

  1. Introduction
  2. Revisiting ChebNet
  3. Spectral-based GNNs and ChebNet
  4. The motivation of revisiting ChebNet
  5. Coefficient Constraints
  6. ChebNetII model
  7. Chebyshev interpolation
  8. ChebNetII via Chebyshev Interpolation
  9. Analysis of ChebNetII
  10. Experiments
  11. Semi-supervised node classification with polynomial-based methods
  12. Full-supervised node classification
  13. Scalability of ChebNetII
  14. Comparison of Different Polynomial Bases
  15. Conclusion
  16. ...and 10 more sections

Key Result

Theorem 2.1

zhang2021asymptotic If $f(x)$ is weakly singular at the boundaries and analytic in the interval $(-1,1)$, then the Chebyshev coefficients $w_k$ will asymptotically (as $k \to \infty$) decrease proportionally to $1/k^q$ for some positive constant $q$.

Figures (6)

  • Figure 1: Illustrations of the Chebyshev expansion's coefficients of $exp({\hat{\lambda}})$ and the Chebyshev coefficients learnt by ChebBase and ChebBase/$k$ on Cora.
  • Figure 2: (a) A Runge filter $h(\Hat{\lambda})=1/(1+25\Hat{\lambda}^2)$ and its approximation results by different polynomial bases. (b) The errors of the different approximation results.
  • Figure 3: A ring graph and filtering results. The red bar indicates the graph signal and the blue line charts represent the eigenvectors. Node labels are represented by orange and purple colors.
  • Figure 4: Chebyshev polynomials $T_k(x), k=0,1,2,3,4,5$.
  • Figure 5: Illustrations of the Chebyshev coefficients learnt by ChebBase and ChebBase/$k$ on Citeseer and Pubmed.
  • ...and 1 more figures

Theorems & Definitions (8)

  • Theorem 2.1
  • Definition 3.1
  • Definition 3.2
  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Lemma D.1
  • Lemma D.2