Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One-Shot Multi-Rate Pruning of Graph Convolutional Networks

Hichem Sahbi

TL;DR

MRMP presents a variational, band-stop based pruning framework for Graph Convolutional Networks that jointly learns topology and weights by constraining latent weight distributions to a fixed prior. It introduces SRMP for single-rate pruning and MRMP for multi-rate pruning with shared latent weights, enabling accurate networks at unseen pruning rates without retraining. The method leverages a KLD regularizer $D_{KL}(P||Q)$ and a quantile-based mapping $a(r)$ to control pruning budget, yielding strong performance on skeleton-based recognition benchmarks. Practically, MRMP offers efficient, scalable pruning with improved generalization and instant inference across a continuum of pruning rates, useful for edge deployments.

Abstract

In this paper, we devise a novel lightweight Graph Convolutional Network (GCN) design dubbed as Multi-Rate Magnitude Pruning (MRMP) that jointly trains network topology and weights. Our method is variational and proceeds by aligning the weight distribution of the learned networks with an a priori distribution. In the one hand, this allows implementing any fixed pruning rate, and also enhancing the generalization performances of the designed lightweight GCNs. In the other hand, MRMP achieves a joint training of multiple GCNs, on top of shared weights, in order to extrapolate accurate networks at any targeted pruning rate without retraining their weights. Extensive experiments conducted on the challenging task of skeleton-based recognition show a substantial gain of our lightweight GCNs particularly at very high pruning regimes.

One-Shot Multi-Rate Pruning of Graph Convolutional Networks

TL;DR

MRMP presents a variational, band-stop based pruning framework for Graph Convolutional Networks that jointly learns topology and weights by constraining latent weight distributions to a fixed prior. It introduces SRMP for single-rate pruning and MRMP for multi-rate pruning with shared latent weights, enabling accurate networks at unseen pruning rates without retraining. The method leverages a KLD regularizer and a quantile-based mapping to control pruning budget, yielding strong performance on skeleton-based recognition benchmarks. Practically, MRMP offers efficient, scalable pruning with improved generalization and instant inference across a continuum of pruning rates, useful for edge deployments.

Abstract

In this paper, we devise a novel lightweight Graph Convolutional Network (GCN) design dubbed as Multi-Rate Magnitude Pruning (MRMP) that jointly trains network topology and weights. Our method is variational and proceeds by aligning the weight distribution of the learned networks with an a priori distribution. In the one hand, this allows implementing any fixed pruning rate, and also enhancing the generalization performances of the designed lightweight GCNs. In the other hand, MRMP achieves a joint training of multiple GCNs, on top of shared weights, in order to extrapolate accurate networks at any targeted pruning rate without retraining their weights. Extensive experiments conducted on the challenging task of skeleton-based recognition show a substantial gain of our lightweight GCNs particularly at very high pruning regimes.
Paper Structure (7 sections, 5 equations, 4 figures, 4 tables)

This paper contains 7 sections, 5 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Proposed Method
  4. SRMP: Single-Rate Magnitude Pruning
  5. MRMP: Multi-Rate Magnitude Pruning
  6. Experiments
  7. Conclusion

Figures (4)

  • Figure 1: This figure shows a Band-stop function $\psi_{a,\sigma}$ applied to a given (gaussian) weight distribution. Depending on $a$, only weights with large magnitudes are kept (Better to zoom the file).
  • Figure 2: From left to right: the 3 figures correspond to targeted (uniform, gaussian and laplace) distributions, and the 4th figure shows the actual weight distribution of the unpruned GCN which resembles to gaussian/laplacian.
  • Figure 3: This figure shows original skeletons (left) on the SBU and (middle) the FPHA datasets. (Right) this figure shows the whole keypoint tracking and description process.
  • Figure 4: Performances on the SBU dataset: (Top) Fixed and observed pruning rates when different PDFs are used in the KLD regularizer. (Bottom) Performances for different (seen and unseen) pruning rates of MRMP; again seen pruning rates (during training) correspond to 50, 55, 60, 65, 70, 75, 80, 85, 90, 95 and 98% while unseen ones correspond to all the remaining pruning rates in $[50,100[$. Better to zoom the file.