Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GNNavigator: Towards Adaptive Training of Graph Neural Networks via Automatic Guideline Exploration

Tong Qiao, Jianlei Yang, Yingjie Qi, Ao Zhou, Chen Bai, Bei Yu, Weisheng Zhao, Chunming Hu

TL;DR

GNNavigator tackles the challenge of balancing training time, memory usage, and accuracy in graph neural networks across diverse applications and heterogeneous hardware. It introduces a unified abstraction of training optimizations, a reconfigurable runtime backend, and a gray-box performance estimator to enable automated, application-driven design-space exploration that generates adaptive training guidelines. The framework can reproduce or surpass existing optimization strategies, delivering up to 3.1x speedups and up to 44.9% memory reductions with comparable accuracy. By validating on multiple datasets and models, GNNavigator demonstrates strong adaptability to different performance priorities and practical impact for scalable GNN training in real-world settings.

Abstract

Graph Neural Networks (GNNs) succeed significantly in many applications recently. However, balancing GNNs training runtime cost, memory consumption, and attainable accuracy for various applications is non-trivial. Previous training methodologies suffer from inferior adaptability and lack a unified training optimization solution. To address the problem, this work proposes GNNavigator, an adaptive GNN training configuration optimization framework. GNNavigator meets diverse GNN application requirements due to our unified software-hardware co-abstraction, proposed GNNs training performance model, and practical design space exploration solution. Experimental results show that GNNavigator can achieve up to 3.1x speedup and 44.9% peak memory reduction with comparable accuracy to state-of-the-art approaches.

GNNavigator: Towards Adaptive Training of Graph Neural Networks via Automatic Guideline Exploration

TL;DR

GNNavigator tackles the challenge of balancing training time, memory usage, and accuracy in graph neural networks across diverse applications and heterogeneous hardware. It introduces a unified abstraction of training optimizations, a reconfigurable runtime backend, and a gray-box performance estimator to enable automated, application-driven design-space exploration that generates adaptive training guidelines. The framework can reproduce or surpass existing optimization strategies, delivering up to 3.1x speedups and up to 44.9% memory reductions with comparable accuracy. By validating on multiple datasets and models, GNNavigator demonstrates strong adaptability to different performance priorities and practical impact for scalable GNN training in real-world settings.

Abstract

Graph Neural Networks (GNNs) succeed significantly in many applications recently. However, balancing GNNs training runtime cost, memory consumption, and attainable accuracy for various applications is non-trivial. Previous training methodologies suffer from inferior adaptability and lack a unified training optimization solution. To address the problem, this work proposes GNNavigator, an adaptive GNN training configuration optimization framework. GNNavigator meets diverse GNN application requirements due to our unified software-hardware co-abstraction, proposed GNNs training performance model, and practical design space exploration solution. Experimental results show that GNNavigator can achieve up to 3.1x speedup and 44.9% peak memory reduction with comparable accuracy to state-of-the-art approaches.
Paper Structure (15 sections, 12 equations, 6 figures, 2 tables, 1 algorithm)

This paper contains 15 sections, 12 equations, 6 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Motivations
  3. Mini-batch based GNNs Training
  4. Heterogeneous Platforms for GNNs Training
  5. Observations and Opportunities
  6. GNNavigator Framework
  7. Framework Overview
  8. Unified Abstraction of Training Optimizations
  9. Automatic Guidelines Generation
  10. Experimental Results
  11. Evaluation Setup
  12. Overall Performance
  13. Impact of Application Adaption
  14. Precision of Performance Estimator
  15. Conclusions

Figures (6)

  • Figure 1: Profiling on existing GNN training frameworks.
  • Figure 2: Framework overview of GNNavigator.
  • Figure 3: Reconfigurable runtime backend of GNNavigator.
  • Figure 4: Automatic guidelines exploration.
  • Figure 5: Accuracy comparison between different estimator models. The distance between each blue point and the red dash-line reflects the estimator’s accuracy
  • ...and 1 more figures