Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Acceleration Algorithms in GNNs: A Survey

Lu Ma, Zeang Sheng, Xunkai Li, Xinyi Gao, Zhezheng Hao, Ling Yang, Wentao Zhang, Bin Cui

TL;DR

This survey addresses the inefficiencies of training and inference in Graph Neural Networks on large-scale graphs by structuring acceleration techniques into training, inference, and execution categories. It highlights representative methods such as graph sampling and GNN simplification for training, knowledge distillation, quantization, and pruning for inference, and binarization plus graph condensation for execution, while also surveying acceleration-focused libraries like SGL. The work characterizes each approach's trade-offs in time and memory, and discusses how these methods can be integrated into scalable pipelines. By outlining a coherent taxonomy and practical avenues, the paper aims to guide scalable GNN deployment in real-world, large-graph scenarios and points toward promising future directions such as complex graph types and DC-GML integration.

Abstract

Graph Neural Networks (GNNs) have demonstrated effectiveness in various graph-based tasks. However, their inefficiency in training and inference presents challenges for scaling up to real-world and large-scale graph applications. To address the critical challenges, a range of algorithms have been proposed to accelerate training and inference of GNNs, attracting increasing attention from the research community. In this paper, we present a systematic review of acceleration algorithms in GNNs, which can be categorized into three main topics based on their purpose: training acceleration, inference acceleration, and execution acceleration. Specifically, we summarize and categorize the existing approaches for each main topic, and provide detailed characterizations of the approaches within each category. Additionally, we review several libraries related to acceleration algorithms in GNNs and discuss our Scalable Graph Learning (SGL) library. Finally, we propose promising directions for future research. A complete summary is presented in our GitHub repository: https://github.com/PKU-DAIR/SGL/blob/main/Awsome-GNN-Acceleration.md.

Acceleration Algorithms in GNNs: A Survey

TL;DR

This survey addresses the inefficiencies of training and inference in Graph Neural Networks on large-scale graphs by structuring acceleration techniques into training, inference, and execution categories. It highlights representative methods such as graph sampling and GNN simplification for training, knowledge distillation, quantization, and pruning for inference, and binarization plus graph condensation for execution, while also surveying acceleration-focused libraries like SGL. The work characterizes each approach's trade-offs in time and memory, and discusses how these methods can be integrated into scalable pipelines. By outlining a coherent taxonomy and practical avenues, the paper aims to guide scalable GNN deployment in real-world, large-graph scenarios and points toward promising future directions such as complex graph types and DC-GML integration.

Abstract

Graph Neural Networks (GNNs) have demonstrated effectiveness in various graph-based tasks. However, their inefficiency in training and inference presents challenges for scaling up to real-world and large-scale graph applications. To address the critical challenges, a range of algorithms have been proposed to accelerate training and inference of GNNs, attracting increasing attention from the research community. In this paper, we present a systematic review of acceleration algorithms in GNNs, which can be categorized into three main topics based on their purpose: training acceleration, inference acceleration, and execution acceleration. Specifically, we summarize and categorize the existing approaches for each main topic, and provide detailed characterizations of the approaches within each category. Additionally, we review several libraries related to acceleration algorithms in GNNs and discuss our Scalable Graph Learning (SGL) library. Finally, we propose promising directions for future research. A complete summary is presented in our GitHub repository: https://github.com/PKU-DAIR/SGL/blob/main/Awsome-GNN-Acceleration.md.
Paper Structure (16 sections, 7 equations, 3 figures, 1 table)

This paper contains 16 sections, 7 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Graph Neural Networks
  4. Challenges of scalable GNNs
  5. Training Acceleration
  6. Graph Sampling
  7. GNN Simplification
  8. GNN Inference Acceleration
  9. Knowledge Distillation
  10. GNN Quantization
  11. GNN Pruning
  12. GNN Execution Acceleration
  13. GNN Binarization
  14. Graph Condensation
  15. Libraries for GNNs Acceleration
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Taxonomy of GNN acceleration algorithms. We comprehensively introduce GNNs training acceleration in Section 3, GNNs inference acceleration in Section 4 and GNNs execution acceleration in Section 5, followed by our discussions of related libraries in Section 6 and future research directions in Section 7.
  • Figure 2: Illustration of SGAP for GNN simplification methods.
  • Figure 3: The architecture of SGL framework