Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerating PoT Quantization on Edge Devices

Rappy Saha, Jude Haris, José Cano

TL;DR

This paper designs a shift-based accelerator using the most efficient shift-PE and proposes PoTAcc, an open-source pipeline for end-to-end acceleration of PoT-quantized DNNs on resource-constrained edge devices.

Abstract

Non-uniform quantization, such as power-of-two (PoT) quantization, matches data distributions better than uniform quantization, which reduces the quantization error of Deep Neural Networks (DNNs). PoT quantization also allows bit-shift operations to replace multiplications, but there are limited studies on the efficiency of shift-based accelerators for PoT quantization. Furthermore, existing pipelines for accelerating PoT-quantized DNNs on edge devices are not open-source. In this paper, we first design shift-based processing elements (shift-PE) for different PoT quantization methods and evaluate their efficiency using synthetic benchmarks. Then we design a shift-based accelerator using our most efficient shift-PE and propose PoTAcc, an open-source pipeline for end-to-end acceleration of PoT-quantized DNNs on resource-constrained edge devices. Using PoTAcc, we evaluate the performance of our shift-based accelerator across three DNNs. On average, it achieves a 1.23x speedup and 1.24x energy reduction compared to a multiplier-based accelerator, and a 2.46x speedup and 1.83x energy reduction compared to CPU-only execution. Our code is available at https://github.com/gicLAB/PoTAcc

Accelerating PoT Quantization on Edge Devices

TL;DR

This paper designs a shift-based accelerator using the most efficient shift-PE and proposes PoTAcc, an open-source pipeline for end-to-end acceleration of PoT-quantized DNNs on resource-constrained edge devices.

Abstract

Non-uniform quantization, such as power-of-two (PoT) quantization, matches data distributions better than uniform quantization, which reduces the quantization error of Deep Neural Networks (DNNs). PoT quantization also allows bit-shift operations to replace multiplications, but there are limited studies on the efficiency of shift-based accelerators for PoT quantization. Furthermore, existing pipelines for accelerating PoT-quantized DNNs on edge devices are not open-source. In this paper, we first design shift-based processing elements (shift-PE) for different PoT quantization methods and evaluate their efficiency using synthetic benchmarks. Then we design a shift-based accelerator using our most efficient shift-PE and propose PoTAcc, an open-source pipeline for end-to-end acceleration of PoT-quantized DNNs on resource-constrained edge devices. Using PoTAcc, we evaluate the performance of our shift-based accelerator across three DNNs. On average, it achieves a 1.23x speedup and 1.24x energy reduction compared to a multiplier-based accelerator, and a 2.46x speedup and 1.83x energy reduction compared to CPU-only execution. Our code is available at https://github.com/gicLAB/PoTAcc
Paper Structure (17 sections, 1 equation, 2 figures, 4 tables)

This paper contains 17 sections, 1 equation, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Power-of-Two Quantization
  4. Processing Element Design for PoT Quantization
  5. Pipelines for PoT Quantization Acceleration
  6. Hardware Analysis of PoT Quantization
  7. PoT Quantization Methods
  8. Shift-PE Design
  9. Hardware Efficiency of Shift-PEs
  10. PoTAcc Pipeline
  11. DNN Model Generation
  12. Accelerator Design and Inference
  13. Evaluation
  14. Experimental Setup
  15. Synthetic Benchmarks
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Shift-PE design: (a) 8_4_pot_QKeras; (b) 8_4_pot_MSQ; (c) 8_4_pot_APoT.
  • Figure 2: PoTAcc: End-to-End Pipeline for PoT Quantization Acceleration.