Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Designing Semi-Structured Pruning of Graph Convolutional Networks for Skeleton-based Recognition

Hichem Sahbi

TL;DR

This work tackles the need for compact graph convolutional networks for skeleton-based recognition on edge devices by introducing semi-structured magnitude pruning. It leverages a differentiable cascaded parametrization with band-stop, weight-sharing, and gating to jointly learn pruned topology and weights without doubling parameters, guided by a budget-aware loss that enforces a target pruning budget $c$ and uses a differentiable rank surrogate $r(W)$. The approach yields a flexible spectrum from highly structured to unstructured pruning, delivering substantial speedups while preserving accuracy, and outperforms purely structured/unstructured pruning and other regularizers on SBU and FPHA benchmarks. The results demonstrate the practicality of deploying efficient GCNs for gesture and action recognition in resource-constrained environments.

Abstract

Deep neural networks (DNNs) are nowadays witnessing a major success in solving many pattern recognition tasks including skeleton-based classification. The deployment of DNNs on edge-devices, endowed with limited time and memory resources, requires designing lightweight and efficient variants of these networks. Pruning is one of the lightweight network design techniques that operate by removing unnecessary network parts, in a structured or an unstructured manner, including individual weights, neurons or even entire channels. Nonetheless, structured and unstructured pruning methods, when applied separately, may either be inefficient or ineffective. In this paper, we devise a novel semi-structured method that discards the downsides of structured and unstructured pruning while gathering their upsides to some extent. The proposed solution is based on a differentiable cascaded parametrization which combines (i) a band-stop mechanism that prunes weights depending on their magnitudes, (ii) a weight-sharing parametrization that prunes connections either individually or group-wise, and (iii) a gating mechanism which arbitrates between different group-wise and entry-wise pruning. All these cascaded parametrizations are built upon a common latent tensor which is trained end-to-end by minimizing a classification loss and a surrogate tensor rank regularizer. Extensive experiments, conducted on the challenging tasks of action and hand-gesture recognition, show the clear advantage of our proposed semi-structured pruning approach against both structured and unstructured pruning, when taken separately, as well as the related work.

Designing Semi-Structured Pruning of Graph Convolutional Networks for Skeleton-based Recognition

TL;DR

This work tackles the need for compact graph convolutional networks for skeleton-based recognition on edge devices by introducing semi-structured magnitude pruning. It leverages a differentiable cascaded parametrization with band-stop, weight-sharing, and gating to jointly learn pruned topology and weights without doubling parameters, guided by a budget-aware loss that enforces a target pruning budget and uses a differentiable rank surrogate . The approach yields a flexible spectrum from highly structured to unstructured pruning, delivering substantial speedups while preserving accuracy, and outperforms purely structured/unstructured pruning and other regularizers on SBU and FPHA benchmarks. The results demonstrate the practicality of deploying efficient GCNs for gesture and action recognition in resource-constrained environments.

Abstract

Deep neural networks (DNNs) are nowadays witnessing a major success in solving many pattern recognition tasks including skeleton-based classification. The deployment of DNNs on edge-devices, endowed with limited time and memory resources, requires designing lightweight and efficient variants of these networks. Pruning is one of the lightweight network design techniques that operate by removing unnecessary network parts, in a structured or an unstructured manner, including individual weights, neurons or even entire channels. Nonetheless, structured and unstructured pruning methods, when applied separately, may either be inefficient or ineffective. In this paper, we devise a novel semi-structured method that discards the downsides of structured and unstructured pruning while gathering their upsides to some extent. The proposed solution is based on a differentiable cascaded parametrization which combines (i) a band-stop mechanism that prunes weights depending on their magnitudes, (ii) a weight-sharing parametrization that prunes connections either individually or group-wise, and (iii) a gating mechanism which arbitrates between different group-wise and entry-wise pruning. All these cascaded parametrizations are built upon a common latent tensor which is trained end-to-end by minimizing a classification loss and a surrogate tensor rank regularizer. Extensive experiments, conducted on the challenging tasks of action and hand-gesture recognition, show the clear advantage of our proposed semi-structured pruning approach against both structured and unstructured pruning, when taken separately, as well as the related work.

Paper Structure

This paper contains 9 sections, 8 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. A Glimpse on Graph Convolutional Networks
  4. Proposed Method: Semi-Structured Magnitude Pruning
  5. Semi-Structured Weight Parametrization
  6. Budget-Aware Variational Pruning
  7. Optimization
  8. Experiments
  9. Conclusion

Figures (2)

  • Figure 1: This figure shows the three stages of the cascaded parametrization including (i) band-stop, (ii) weight-sharing and (iii) gating. Cyan stands for shared connections, and the triangle for the "not gate" operator. For ease of visualization, only 4 connections are shown during the whole evaluation of the parameterization, and only the outcome (1 or 0) of $w_{i,j}$ is shown in the final mask tensor.
  • Figure 2: This figure shows a crop of the mask tensor obtained after the gating parametrization when trained on the FPHA dataset. Top-left corresponds to the original mask (without pruning) while the others correspond to masks obtained with structured, unstructured and semi-structured pruning respectively. In all these masks, each diagonal block corresponds to a channel. Better to zoom the PDF.