Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Joint Pruning and Channel-wise Mixed-Precision Quantization for Efficient Deep Neural Networks

Beatrice Alessandra Motetti, Matteo Risso, Alessio Burrello, Enrico Macii, Massimo Poncino, Daniele Jahier Pagliari

TL;DR

A novel methodology to apply state-of-the-art pruning and mixed-precision optimizations jointly via a lightweight gradient-based search, and in a hardware-aware manner, greatly reducing the time required to generate Pareto-optimal DNNs in terms of accuracy versus cost.

Abstract

The resource requirements of deep neural networks (DNNs) pose significant challenges to their deployment on edge devices. Common approaches to address this issue are pruning and mixed-precision quantization, which lead to latency and memory occupation improvements. These optimization techniques are usually applied independently. We propose a novel methodology to apply them jointly via a lightweight gradient-based search, and in a hardware-aware manner, greatly reducing the time required to generate Pareto-optimal DNNs in terms of accuracy versus cost (i.e., latency or memory). We test our approach on three edge-relevant benchmarks, namely CIFAR-10, Google Speech Commands, and Tiny ImageNet. When targeting the optimization of the memory footprint, we are able to achieve a size reduction of 47.50% and 69.54% at iso-accuracy with the baseline networks with all weights quantized at 8 and 2-bit, respectively. Our method surpasses a previous state-of-the-art approach with up to 56.17% size reduction at iso-accuracy. With respect to the sequential application of state-of-the-art pruning and mixed-precision optimizations, we obtain comparable or superior results, but with a significantly lowered training time. In addition, we show how well-tailored cost models can improve the cost versus accuracy trade-offs when targeting specific hardware for deployment.

Joint Pruning and Channel-wise Mixed-Precision Quantization for Efficient Deep Neural Networks

TL;DR

A novel methodology to apply state-of-the-art pruning and mixed-precision optimizations jointly via a lightweight gradient-based search, and in a hardware-aware manner, greatly reducing the time required to generate Pareto-optimal DNNs in terms of accuracy versus cost.

Abstract

The resource requirements of deep neural networks (DNNs) pose significant challenges to their deployment on edge devices. Common approaches to address this issue are pruning and mixed-precision quantization, which lead to latency and memory occupation improvements. These optimization techniques are usually applied independently. We propose a novel methodology to apply them jointly via a lightweight gradient-based search, and in a hardware-aware manner, greatly reducing the time required to generate Pareto-optimal DNNs in terms of accuracy versus cost (i.e., latency or memory). We test our approach on three edge-relevant benchmarks, namely CIFAR-10, Google Speech Commands, and Tiny ImageNet. When targeting the optimization of the memory footprint, we are able to achieve a size reduction of 47.50% and 69.54% at iso-accuracy with the baseline networks with all weights quantized at 8 and 2-bit, respectively. Our method surpasses a previous state-of-the-art approach with up to 56.17% size reduction at iso-accuracy. With respect to the sequential application of state-of-the-art pruning and mixed-precision optimizations, we obtain comparable or superior results, but with a significantly lowered training time. In addition, we show how well-tailored cost models can improve the cost versus accuracy trade-offs when targeting specific hardware for deployment.
Paper Structure (33 sections, 13 equations, 9 figures, 3 tables)

This paper contains 33 sections, 13 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Introduction
  3. Background
  4. Quantization and Mixed-Precision Search
  5. Pruning
  6. Related Works
  7. Mixed-Precision Search
  8. Non-differentiable techniques
  9. Differentiable techniques
  10. Joint Quantization and Pruning
  11. Non-differentiable techniques
  12. Differentiable techniques
  13. Proposed Method
  14. Search Space
  15. Optimization Method
  16. ...and 18 more sections

Figures (9)

  • Figure 1: Overview of the key components of the proposed method, with the required inputs (yellow), the core optimization scheme (red), and the final result, i.e. a pruned mixed-precision DNN (green).
  • Figure 2: Overview of the quantization step of our proposed approach
  • Figure 3: Reordering of the weights channels by bit-width after the precision assignment
  • Figure 4: Results obtained with our proposed approach on CIFAR-10, GSC and Tiny ImageNet. Different sampling methods are compared.
  • Figure 5: Comparison of the results obtained by our proposed method with other state-of-the-art approaches. The architecture optimized by PIT that is used as input for MixPrec is denoted as PIT Seed.
  • ...and 4 more figures