Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Pruning and Quantization Impact on Graph Neural Networks

Khatoon Khedri, Reza Rawassizadeh, Qifu Wen, Mehdi Hosseinzadeh

TL;DR

This work investigates neural network compression for Graph Neural Networks by examining three pruning methods (fine-grained pruning, global pruning, and L2 regularization) and three quantization methods ($A^2Q$, QAT, and DQ) across Cora, Proteins, and BBBP. Using state-of-the-art GNN architectures, the study demonstrates that unstructured pruning can cut model size by about 50% with accuracy preserved or improved after fine-tuning, while quantization offers mixed but often favorable gains in accuracy, speed, and memory depending on dataset and method. A key finding is that pruning alone does not reliably reduce inference latency or memory usage, highlighting the need for task- and layer-specific strategies; in contrast, quantization methods like $QAT$-$INT8$ and $DQ$-$INT8$ can deliver competitive accuracy with notable computational savings. Additionally, the results challenge the lottery ticket hypothesis for GNNs, suggesting that smaller, equally performant sub-networks may not readily exist in graph-based models. Overall, the paper provides practical guidance on combining pruning and quantization to achieve efficient, high-accuracy GNNs for edge- and resource-constrained deployments.

Abstract

Graph neural networks (GNNs) are known to operate with high accuracy on learning from graph-structured data, but they suffer from high computational and resource costs. Neural network compression methods are used to reduce the model size while maintaining reasonable accuracy. Two of the common neural network compression techniques include pruning and quantization. In this research, we empirically examine the effects of three pruning methods and three quantization methods on different GNN models, including graph classification tasks, node classification tasks, and link prediction. We conducted all experiments on three graph datasets, including Cora, Proteins, and BBBP. Our findings demonstrate that unstructured fine-grained and global pruning can significantly reduce the model's size(50\%) while maintaining or even improving precision after fine-tuning the pruned model. The evaluation of different quantization methods on GNN shows diverse impacts on accuracy, inference time, and model size across different datasets.

Pruning and Quantization Impact on Graph Neural Networks

TL;DR

This work investigates neural network compression for Graph Neural Networks by examining three pruning methods (fine-grained pruning, global pruning, and L2 regularization) and three quantization methods (, QAT, and DQ) across Cora, Proteins, and BBBP. Using state-of-the-art GNN architectures, the study demonstrates that unstructured pruning can cut model size by about 50% with accuracy preserved or improved after fine-tuning, while quantization offers mixed but often favorable gains in accuracy, speed, and memory depending on dataset and method. A key finding is that pruning alone does not reliably reduce inference latency or memory usage, highlighting the need for task- and layer-specific strategies; in contrast, quantization methods like - and - can deliver competitive accuracy with notable computational savings. Additionally, the results challenge the lottery ticket hypothesis for GNNs, suggesting that smaller, equally performant sub-networks may not readily exist in graph-based models. Overall, the paper provides practical guidance on combining pruning and quantization to achieve efficient, high-accuracy GNNs for edge- and resource-constrained deployments.

Abstract

Graph neural networks (GNNs) are known to operate with high accuracy on learning from graph-structured data, but they suffer from high computational and resource costs. Neural network compression methods are used to reduce the model size while maintaining reasonable accuracy. Two of the common neural network compression techniques include pruning and quantization. In this research, we empirically examine the effects of three pruning methods and three quantization methods on different GNN models, including graph classification tasks, node classification tasks, and link prediction. We conducted all experiments on three graph datasets, including Cora, Proteins, and BBBP. Our findings demonstrate that unstructured fine-grained and global pruning can significantly reduce the model's size(50\%) while maintaining or even improving precision after fine-tuning the pruned model. The evaluation of different quantization methods on GNN shows diverse impacts on accuracy, inference time, and model size across different datasets.
Paper Structure (23 sections, 10 figures, 17 tables, 1 algorithm)

This paper contains 23 sections, 10 figures, 17 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Experimental Setup
  4. Datasets
  5. Models and Down Stream Tasks
  6. Pruning Methods
  7. Pruned Methods Model Size using Sparse Model State Compression Algorithm
  8. Quantization Methods
  9. QAT, and DQ:
  10. $A^{2}Q$: Aggregation-aware Quantization:
  11. Experimental Evaluation
  12. Setup Experiments
  13. Pruning
  14. Layers Sparsity Through Global Pruning
  15. The Impact of Pruning on Convergence
  16. ...and 8 more sections

Figures (10)

  • Figure 1: Sparsity of 50$\%$ layer$conv1(spline)$ of pruning of SPLINE model for node classification task of Cora dataset.
  • Figure 2: Layer-wise sparsity tolerance across three architectures: GNNNets on BBBP, Spline on Cora, and HGPSL on Proteins. Each segmented line represents a layer, with colors indicating sparsity levels from $0.0$ (dense, no pruning) to $0.9$($90\%$ weights removed).
  • Figure 3: The Changing Accuracy using pruning methods
  • Figure 4: The Changing Memory Usage through implementing Pruning Methods
  • Figure 5: The Changing energy consumption of Using Pruning Methods
  • ...and 5 more figures