Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards a tailored mixed-precision sub-8-bit quantization scheme for Gated Recurrent Units using Genetic Algorithms

Riccardo Miccini, Alessandro Cerioli, Clément Laroche, Tobias Piechowiak, Jens Sparsø, Luca Pezzarossa

TL;DR

The paper tackles the challenge of deploying GRU-based models on ultra-low-power devices by addressing sub-8-bit quantization. It introduces a modular, integer-only mixed-precision quantization scheme where each GRU operation has an independently chosen bit-width, and it employs NSGA-II genetic algorithms to jointly optimize accuracy and model size. The authors demonstrate that mixed-precision schemes yield better Pareto efficiency than homogeneous quantization across four sequential tasks, achieving 25-55% model-size reductions while maintaining comparable accuracy to 8-bit baselines. They also reveal task-specific quantization patterns via genome clustering, underscoring the value of bespoke bit-width configurations for GRUs in embedded settings.

Abstract

Despite the recent advances in model compression techniques for deep neural networks, deploying such models on ultra-low-power embedded devices still proves challenging. In particular, quantization schemes for Gated Recurrent Units (GRU) are difficult to tune due to their dependence on an internal state, preventing them from fully benefiting from sub-8bit quantization. In this work, we propose a modular integer quantization scheme for GRUs where the bit width of each operator can be selected independently. We then employ Genetic Algorithms (GA) to explore the vast search space of possible bit widths, simultaneously optimising for model size and accuracy. We evaluate our methods on four different sequential tasks and demonstrate that mixed-precision solutions exceed homogeneous-precision ones in terms of Pareto efficiency. In our results, we achieve a model size reduction between 25% and 55% while maintaining an accuracy comparable with the 8-bit homogeneous equivalent.

Towards a tailored mixed-precision sub-8-bit quantization scheme for Gated Recurrent Units using Genetic Algorithms

TL;DR

The paper tackles the challenge of deploying GRU-based models on ultra-low-power devices by addressing sub-8-bit quantization. It introduces a modular, integer-only mixed-precision quantization scheme where each GRU operation has an independently chosen bit-width, and it employs NSGA-II genetic algorithms to jointly optimize accuracy and model size. The authors demonstrate that mixed-precision schemes yield better Pareto efficiency than homogeneous quantization across four sequential tasks, achieving 25-55% model-size reductions while maintaining comparable accuracy to 8-bit baselines. They also reveal task-specific quantization patterns via genome clustering, underscoring the value of bespoke bit-width configurations for GRUs in embedded settings.

Abstract

Despite the recent advances in model compression techniques for deep neural networks, deploying such models on ultra-low-power embedded devices still proves challenging. In particular, quantization schemes for Gated Recurrent Units (GRU) are difficult to tune due to their dependence on an internal state, preventing them from fully benefiting from sub-8bit quantization. In this work, we propose a modular integer quantization scheme for GRUs where the bit width of each operator can be selected independently. We then employ Genetic Algorithms (GA) to explore the vast search space of possible bit widths, simultaneously optimising for model size and accuracy. We evaluate our methods on four different sequential tasks and demonstrate that mixed-precision solutions exceed homogeneous-precision ones in terms of Pareto efficiency. In our results, we achieve a model size reduction between 25% and 55% while maintaining an accuracy comparable with the 8-bit homogeneous equivalent.
Paper Structure (19 sections, 14 equations, 5 figures, 1 table)

This paper contains 19 sections, 14 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Gated Recurrent Unit
  4. Quantization
  5. Genetic Algorithms
  6. Modular quantization
  7. Quantization scheme for GRU
  8. Linear layer
  9. Element-wise sum
  10. Element-wise product
  11. Activations
  12. Quantization scheme search
  13. Experimental evaluation
  14. Setup
  15. Sequential tasks
  16. ...and 4 more sections

Figures (5)

  • Figure 1: A GRU with its building blocks exposed; the different colors denote the operations associated with \ref{['eq:gru']}.
  • Figure 2: Typical flow of a genetic algorithm, with red blocks indicating the custom evaluation pipeline.
  • Figure 3: Enhanced box plot showing the distributions of accuracy and model size across generations for the given tasks.
  • Figure 4: Pareto fronts (higher is better) for homogeneous (baseline) vs. mixed-precision (genetic search) quantization; each dot is a solution/individual.
  • Figure 5: Dimensionally-reduced visualization (using t-SNE) of solutions' genomes, colored by accuracy.