Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Simplifying Graph Convolutional Networks

Felix Wu, Tianyi Zhang, Amauri Holanda de Souza, Christopher Fifty, Tao Yu, Kilian Q. Weinberger

TL;DR

GCNs offer strong graph representations but incur complexity and computational costs. The authors introduce Simple Graph Convolution (SGC), a linear model that fixes feature propagation as $S^K$ and trains a single logistic regression on the smoothed features, achieving comparable accuracy with large speedups. Spectral analysis shows SGC implements a fixed low-pass filter on the graph spectrum, with the renormalization trick further shrinking the spectrum to enhance smoothing. Across node classification and diverse downstream tasks, SGC demonstrates competitive performance, high scalability, and interpretability, suggesting it as a robust baseline and a stepping stone toward simpler, more efficient graph learning methods.

Abstract

Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In this paper, we reduce this excess complexity through successively removing nonlinearities and collapsing weight matrices between consecutive layers. We theoretically analyze the resulting linear model and show that it corresponds to a fixed low-pass filter followed by a linear classifier. Notably, our experimental evaluation demonstrates that these simplifications do not negatively impact accuracy in many downstream applications. Moreover, the resulting model scales to larger datasets, is naturally interpretable, and yields up to two orders of magnitude speedup over FastGCN.

Simplifying Graph Convolutional Networks

TL;DR

GCNs offer strong graph representations but incur complexity and computational costs. The authors introduce Simple Graph Convolution (SGC), a linear model that fixes feature propagation as and trains a single logistic regression on the smoothed features, achieving comparable accuracy with large speedups. Spectral analysis shows SGC implements a fixed low-pass filter on the graph spectrum, with the renormalization trick further shrinking the spectrum to enhance smoothing. Across node classification and diverse downstream tasks, SGC demonstrates competitive performance, high scalability, and interpretability, suggesting it as a robust baseline and a stepping stone toward simpler, more efficient graph learning methods.

Abstract

Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In this paper, we reduce this excess complexity through successively removing nonlinearities and collapsing weight matrices between consecutive layers. We theoretically analyze the resulting linear model and show that it corresponds to a fixed low-pass filter followed by a linear classifier. Notably, our experimental evaluation demonstrates that these simplifications do not negatively impact accuracy in many downstream applications. Moreover, the resulting model scales to larger datasets, is naturally interpretable, and yields up to two orders of magnitude speedup over FastGCN.

Paper Structure

This paper contains 41 sections, 4 theorems, 18 equations, 4 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Simple Graph Convolution
  3. Graph Convolutional Networks
  4. Feature propagation
  5. Feature transformation and nonlinear transition.
  6. Classifier.
  7. Simple Graph Convolution
  8. Linearization.
  9. Logistic regression.
  10. Optimization details.
  11. Spectral Analysis
  12. Preliminaries on Graph Convolutions
  13. SGC and Low-Pass Filtering
  14. Related Works
  15. Graph Neural Networks
  16. ...and 26 more sections

Key Result

Theorem 1

Let ${\mathbf{A}}$ be the adjacency matrix of an undirected, weighted, simple graph $\mathcal{G}$ without isolated nodes and with corresponding degree matrix ${\mathbf{D}}$. Let $\tilde{{\mathbf{A}}} = {\mathbf{A}} + \gamma {\mathbf{I}}$, such that $\gamma > 0$, be the augmented adjacency matrix wit

Figures (4)

  • Figure 1: Schematic layout of a GCN v.s. a SGC. Top row: The GCN transforms the feature vectors repeatedly throughout $K$ layers and then applies a linear classifier on the final representation. Bottom row: the SGC reduces the entire procedure to a simple feature propagation step followed by standard logistic regression.
  • Figure 2: Feature (red) and filters (blue) spectral coefficients for different propagation matrices on Cora dataset ($3$rd feature).
  • Figure 3: Performance over training time on Pubmed and Reddit. SGC is the fastest while achieving competitive performance. We are not able to benchmark the training time of GaAN and DGI on Reddit because the implementations are not released.
  • Figure 4: Validation accuracy with SGC using different propagation matrices.

Theorems & Definitions (8)

  • Theorem 1
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • proof : Proof of Theorem 1