Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GSR-GNN: Training Acceleration and Memory-Saving Framework of Deep GNNs on Circuit Graph

Yuebo Luo, Shiyang Li, Yifei Feng, Vishal Kancharla, Shaoyi Huang, Caiwen Ding

Abstract

Graph Neural Networks (GNNs) show strong promise for circuit analysis, but scaling to modern large-scale circuit graphs is limited by GPU memory and training cost, especially for deep models. We revisit deep GNNs for circuit graphs and show that, when trainable, they significantly outperform shallow architectures, motivating an efficient, domain-specific training framework. We propose Grouped-Sparse-Reversible GNN (GSR-GNN), which enables training GNNs with up to hundreds of layers while reducing both compute and memory overhead. GSR-GNN integrates reversible residual modules with a group-wise sparse nonlinear operator that compresses node embeddings without sacrificing task-relevant information, and employs an optimized execution pipeline to eliminate fragmented activation storage and reduce data movement. On sampled circuit graphs, GSR-GNN achieves up to 87.2\% peak memory reduction and over 30$\times$ training speedup with negligible degradation in correlation-based quality metrics, making deep GNNs practical for large-scale EDA workloads.

GSR-GNN: Training Acceleration and Memory-Saving Framework of Deep GNNs on Circuit Graph

Abstract

Graph Neural Networks (GNNs) show strong promise for circuit analysis, but scaling to modern large-scale circuit graphs is limited by GPU memory and training cost, especially for deep models. We revisit deep GNNs for circuit graphs and show that, when trainable, they significantly outperform shallow architectures, motivating an efficient, domain-specific training framework. We propose Grouped-Sparse-Reversible GNN (GSR-GNN), which enables training GNNs with up to hundreds of layers while reducing both compute and memory overhead. GSR-GNN integrates reversible residual modules with a group-wise sparse nonlinear operator that compresses node embeddings without sacrificing task-relevant information, and employs an optimized execution pipeline to eliminate fragmented activation storage and reduce data movement. On sampled circuit graphs, GSR-GNN achieves up to 87.2\% peak memory reduction and over 30 training speedup with negligible degradation in correlation-based quality metrics, making deep GNNs practical for large-scale EDA workloads.

Paper Structure

This paper contains 12 sections, 7 equations, 10 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Problem Statement and Challenges
  4. Challenges for Reversible GNNs
  5. Architecture of GSR-GNN
  6. Operators and Blocks used in GSR-GNN
  7. Enhanced Memory Management
  8. Forward Phase of GSR-GNN
  9. Backward Phase of GSR-GNN
  10. Experiments
  11. Experimental Results
  12. Conclusion

Figures (10)

  • Figure 1: Performance Comparison: RevGNN vs. SAGE vs. GCN on Circuit Graphs 1 and 3 (See Table \ref{['tab:compact_graph_summary']}), where RevGNN outperforms SAGE and GCN and keeps gaining return when increasing depth. Negative scores in Spearman, Kendall, and $R^2$ mean failure to find correlations.
  • Figure 2: Training results of baseline RevGNN on example graph 356-R in Table \ref{['tab:compact_graph_summary']}, where RevGNN pleateaus around 70 layers, where SAGE and GCN are either too low to stay in the range or crashed because of out-of-memory (OOM) error.
  • Figure 3: GPU memory usage and epoch runtime summary of 100-layer RevGNN with 384 hidden channels on Graph 356-RISCY-a1-c5 in Table \ref{['tab:compact_graph_summary']}.
  • Figure 4: Example training time of baseline 100-layer RevGNN on graph 7 in Table \ref{['tab:compact_graph_summary']}. Note that the red 'X' refers to an out-of-memory (OOM) error.
  • Figure 5: Comparison of Conventional graph dataset (upper) and circuit graph 4 (lower), Table \ref{['tab:compact_graph_summary']} in 50-node graph component pattern and node degree distribution. The circuit graph shows an apparently different connecting pattern and node degree distribution from the conventional graph.
  • ...and 5 more figures