Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large-Scale Spectral Graph Neural Networks via Laplacian Sparsification: Technical Report

Haipeng Ding, Zhewei Wei, Yuhang Ye

TL;DR

This work tackles the scalability barrier of spectral GNNs by introducing SGNN-LS, a Laplacian sparsification framework that approximates the propagation operator L_K=∑_{k=0}^K w_k L^k with a sparse graph while preserving end-to-end training. It provides rigorous guarantees: the sparsifier achieves spectral similarity within a tolerance ε with high probability and uses O(n log n / ε^2) edges, with the propagated signals differing by only O(ε) in the relevant loss. The method supports both static and learnable polynomial coefficients and introduces node-wise sampling for semi-supervised tasks, enabling mini-batch training on very large graphs. Empirical results on large-scale datasets such as Ogbn-papers100M and MAG-scholar-C demonstrate competitive accuracy and substantial memory and time efficiency, confirming the practical impact of the approach and its compatibility with existing scalable GNN strategies.

Abstract

Graph Neural Networks (GNNs) play a pivotal role in graph-based tasks for their proficiency in representation learning. Among the various GNN methods, spectral GNNs employing polynomial filters have shown promising performance on tasks involving both homophilous and heterophilous graph structures. However, The scalability of spectral GNNs on large graphs is limited because they learn the polynomial coefficients through multiple forward propagation executions during forward propagation. Existing works have attempted to scale up spectral GNNs by eliminating the linear layers on the input node features, a change that can disrupt end-to-end training, potentially impact performance, and become impractical with high-dimensional input features. To address the above challenges, we propose "Spectral Graph Neural Networks with Laplacian Sparsification (SGNN-LS)", a novel graph spectral sparsification method to approximate the propagation patterns of spectral GNNs. We prove that our proposed method generates Laplacian sparsifiers that can approximate both fixed and learnable polynomial filters with theoretical guarantees. Our method allows the application of linear layers on the input node features, enabling end-to-end training as well as the handling of raw text features. We conduct an extensive experimental analysis on datasets spanning various graph scales and properties to demonstrate the superior efficiency and effectiveness of our method. The results show that our method yields superior results in comparison with the corresponding approximated base models, especially on dataset Ogbn-papers100M(111M nodes, 1.6B edges) and MAG-scholar-C (2.8M features).

Large-Scale Spectral Graph Neural Networks via Laplacian Sparsification: Technical Report

TL;DR

This work tackles the scalability barrier of spectral GNNs by introducing SGNN-LS, a Laplacian sparsification framework that approximates the propagation operator L_K=∑_{k=0}^K w_k L^k with a sparse graph while preserving end-to-end training. It provides rigorous guarantees: the sparsifier achieves spectral similarity within a tolerance ε with high probability and uses O(n log n / ε^2) edges, with the propagated signals differing by only O(ε) in the relevant loss. The method supports both static and learnable polynomial coefficients and introduces node-wise sampling for semi-supervised tasks, enabling mini-batch training on very large graphs. Empirical results on large-scale datasets such as Ogbn-papers100M and MAG-scholar-C demonstrate competitive accuracy and substantial memory and time efficiency, confirming the practical impact of the approach and its compatibility with existing scalable GNN strategies.

Abstract

Graph Neural Networks (GNNs) play a pivotal role in graph-based tasks for their proficiency in representation learning. Among the various GNN methods, spectral GNNs employing polynomial filters have shown promising performance on tasks involving both homophilous and heterophilous graph structures. However, The scalability of spectral GNNs on large graphs is limited because they learn the polynomial coefficients through multiple forward propagation executions during forward propagation. Existing works have attempted to scale up spectral GNNs by eliminating the linear layers on the input node features, a change that can disrupt end-to-end training, potentially impact performance, and become impractical with high-dimensional input features. To address the above challenges, we propose "Spectral Graph Neural Networks with Laplacian Sparsification (SGNN-LS)", a novel graph spectral sparsification method to approximate the propagation patterns of spectral GNNs. We prove that our proposed method generates Laplacian sparsifiers that can approximate both fixed and learnable polynomial filters with theoretical guarantees. Our method allows the application of linear layers on the input node features, enabling end-to-end training as well as the handling of raw text features. We conduct an extensive experimental analysis on datasets spanning various graph scales and properties to demonstrate the superior efficiency and effectiveness of our method. The results show that our method yields superior results in comparison with the corresponding approximated base models, especially on dataset Ogbn-papers100M(111M nodes, 1.6B edges) and MAG-scholar-C (2.8M features).
Paper Structure (39 sections, 6 theorems, 33 equations, 2 figures, 17 tables, 7 algorithms)

This paper contains 39 sections, 6 theorems, 33 equations, 2 figures, 17 tables, 7 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Proposed Method
  4. Motivation
  5. Simplify the Graph Propagation with Laplacian Sparsification
  6. Laplacian Sparsification for Static Polynomial Coefficients
  7. Laplacian Sparsification for Learnable Polynomial Coefficients
  8. Node-Wise Laplacian Sampling for Semi-supervised Tasks
  9. Theoretical Analysis
  10. Error Guarantee about Laplacian Sparsification
  11. Error Guarantee about Propagated Signals
  12. Related Works
  13. Experiments
  14. Tested Models, Datasets, and Configurations
  15. Accuracy of the Approximation
  16. ...and 24 more sections

Key Result

theorem 1

(Random walk polynomial sparsification.) For any unweighted, undirected graph $G$ with $n$ vertices and $m$ edges, any $\mathbf{w}=[w_0, w_1, ..., w_K]\in\mathcal{R}_+^{(K+1)}, \mathbf{w}\ne\mathbf{0}$ and any approximation parameter $\varepsilon$, we can construct an unbiased $\varepsilon$-sparsifi

Figures (2)

  • Figure 1: An overview of how Laplacian sparsification works. For clarity, the propagation of one single center node is illustrated. Laplacian sparsification is applied to the entire graph, generating fully sparsified graphs to satisfy the propagation requirements of all the nodes in the graph.
  • Figure 2: The visualization of the comparison between the polynomial filtered matrix and the results of the Laplacian sparsification with different numbers of samplings, on dataset Texas.

Theorems & Definitions (7)

  • definition 1
  • theorem 1
  • lemma 1
  • theorem 2
  • proposition 1
  • theorem 3
  • theorem 4