Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Node-wise Propagation for Large-scale Graph Learning

Xunkai Li, Jingyuan Ma, Zhengyu Wu, Daohan Su, Wentao Zhang, Rong-Hua Li, Guoren Wang

TL;DR

Adaptive Topology-aware Propagation (ATP), which reduces potential high-bias propagation and extracts structural patterns of each node in a scalable manner to improve running efficiency and predictive performance and is crafted to be a plug-and-play node-wise propagation optimization strategy.

Abstract

Scalable graph neural networks (GNNs) have emerged as a promising technique, which exhibits superior predictive performance and high running efficiency across numerous large-scale graph-based web applications. However, (i) Most scalable GNNs tend to treat all nodes in graphs with the same propagation rules, neglecting their topological uniqueness; (ii) Existing node-wise propagation optimization strategies are insufficient on web-scale graphs with intricate topology, where a full portrayal of nodes' local properties is required. Intuitively, different nodes in web-scale graphs possess distinct topological roles, and therefore propagating them indiscriminately or neglect local contexts may compromise the quality of node representations. This intricate topology in web-scale graphs cannot be matched by small-scale scenarios. To address the above issues, we propose \textbf{A}daptive \textbf{T}opology-aware \textbf{P}ropagation (ATP), which reduces potential high-bias propagation and extracts structural patterns of each node in a scalable manner to improve running efficiency and predictive performance. Remarkably, ATP is crafted to be a plug-and-play node-wise propagation optimization strategy, allowing for offline execution independent of the graph learning process in a new perspective. Therefore, this approach can be seamlessly integrated into most scalable GNNs while remain orthogonal to existing node-wise propagation optimization strategies. Extensive experiments on 12 datasets, including the most representative large-scale ogbn-papers100M, have demonstrated the effectiveness of ATP. Specifically, ATP has proven to be efficient in improving the performance of prevalent scalable GNNs for semi-supervised node classification while addressing redundant computational costs.

Rethinking Node-wise Propagation for Large-scale Graph Learning

TL;DR

Adaptive Topology-aware Propagation (ATP), which reduces potential high-bias propagation and extracts structural patterns of each node in a scalable manner to improve running efficiency and predictive performance and is crafted to be a plug-and-play node-wise propagation optimization strategy.

Abstract

Scalable graph neural networks (GNNs) have emerged as a promising technique, which exhibits superior predictive performance and high running efficiency across numerous large-scale graph-based web applications. However, (i) Most scalable GNNs tend to treat all nodes in graphs with the same propagation rules, neglecting their topological uniqueness; (ii) Existing node-wise propagation optimization strategies are insufficient on web-scale graphs with intricate topology, where a full portrayal of nodes' local properties is required. Intuitively, different nodes in web-scale graphs possess distinct topological roles, and therefore propagating them indiscriminately or neglect local contexts may compromise the quality of node representations. This intricate topology in web-scale graphs cannot be matched by small-scale scenarios. To address the above issues, we propose \textbf{A}daptive \textbf{T}opology-aware \textbf{P}ropagation (ATP), which reduces potential high-bias propagation and extracts structural patterns of each node in a scalable manner to improve running efficiency and predictive performance. Remarkably, ATP is crafted to be a plug-and-play node-wise propagation optimization strategy, allowing for offline execution independent of the graph learning process in a new perspective. Therefore, this approach can be seamlessly integrated into most scalable GNNs while remain orthogonal to existing node-wise propagation optimization strategies. Extensive experiments on 12 datasets, including the most representative large-scale ogbn-papers100M, have demonstrated the effectiveness of ATP. Specifically, ATP has proven to be efficient in improving the performance of prevalent scalable GNNs for semi-supervised node classification while addressing redundant computational costs.
Paper Structure (22 sections, 3 theorems, 15 equations, 8 figures, 10 tables, 1 algorithm)

This paper contains 22 sections, 3 theorems, 15 equations, 8 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. PRELIMINARIES
  3. Problem Formulation
  4. Scalable Graph Neural Networks
  5. ATP FRAMEWORK
  6. High-bias Propagation Correction
  7. Weight-free LNC Encoding
  8. Experiments
  9. Experimental Setup
  10. Performance Comparison
  11. Ablation Study and In-depth Analysis
  12. Running Efficiency in Large-scale Graphs
  13. Performance under Sparse Graphs
  14. Conclusion
  15. Outline
  16. ...and 7 more sections

Key Result

Lemma 1

The difference between the stable state and $k$-step propagated results represents the upper bound of the convergence rate. where $\tilde{d}$ denotes the degree of node plus 1 (to include itself by self-loop).

Figures (8)

  • Figure 1: Performance in the Cora (2.7k nodes) and ogbn-products (2449k nodes). The x-axis is the training epoch. The red line denotes the baseline performance for all nodes.
  • Figure 2: Predictive performance optimized by ATP.
  • Figure 3: Predictive performance with different kernels.
  • Figure 4: Running times on large-scale graphs.
  • Figure 5: Sparsity performance on ogb-products.
  • ...and 3 more figures

Theorems & Definitions (3)

  • Lemma 1
  • Lemma 2
  • Theorem 1