Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Resource-aware Mixed-precision Quantization for Enhancing Deployability of Transformers for Time-series Forecasting on Embedded FPGAs

Tianheng Ling, Chao Qian, Gregor Schiele

TL;DR

This study enhanced the flexibility of the VHDL template by introducing a selectable resource type for storing intermediate results across model layers, thereby breaking the deployment bottleneck by utilizing BRAM efficiently, and developed a resource-aware mixed-precision quantization approach that enables researchers to explore hardware-level quantization strategies without requiring extensive expertise in Neural Architecture Search.

Abstract

This study addresses the deployment challenges of integer-only quantized Transformers on resource-constrained embedded FPGAs (Xilinx Spartan-7 XC7S15). We enhanced the flexibility of our VHDL template by introducing a selectable resource type for storing intermediate results across model layers, thereby breaking the deployment bottleneck by utilizing BRAM efficiently. Moreover, we developed a resource-aware mixed-precision quantization approach that enables researchers to explore hardware-level quantization strategies without requiring extensive expertise in Neural Architecture Search. This method provides accurate resource utilization estimates with a precision discrepancy as low as 3%, compared to actual deployment metrics. Compared to previous work, our approach has successfully facilitated the deployment of model configurations utilizing mixed-precision quantization, thus overcoming the limitations inherent in five previously non-deployable configurations with uniform quantization bitwidths. Consequently, this research enhances the applicability of Transformers in embedded systems, facilitating a broader range of Transformer-powered applications on edge devices.

Resource-aware Mixed-precision Quantization for Enhancing Deployability of Transformers for Time-series Forecasting on Embedded FPGAs

TL;DR

This study enhanced the flexibility of the VHDL template by introducing a selectable resource type for storing intermediate results across model layers, thereby breaking the deployment bottleneck by utilizing BRAM efficiently, and developed a resource-aware mixed-precision quantization approach that enables researchers to explore hardware-level quantization strategies without requiring extensive expertise in Neural Architecture Search.

Abstract

This study addresses the deployment challenges of integer-only quantized Transformers on resource-constrained embedded FPGAs (Xilinx Spartan-7 XC7S15). We enhanced the flexibility of our VHDL template by introducing a selectable resource type for storing intermediate results across model layers, thereby breaking the deployment bottleneck by utilizing BRAM efficiently. Moreover, we developed a resource-aware mixed-precision quantization approach that enables researchers to explore hardware-level quantization strategies without requiring extensive expertise in Neural Architecture Search. This method provides accurate resource utilization estimates with a precision discrepancy as low as 3%, compared to actual deployment metrics. Compared to previous work, our approach has successfully facilitated the deployment of model configurations utilizing mixed-precision quantization, thus overcoming the limitations inherent in five previously non-deployable configurations with uniform quantization bitwidths. Consequently, this research enhances the applicability of Transformers in embedded systems, facilitating a broader range of Transformer-powered applications on edge devices.
Paper Structure (24 sections, 7 figures, 7 tables, 1 algorithm)

This paper contains 24 sections, 7 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Transformers for Time-series Forecasting
  3. Problem Statement
  4. Proposed Solutions
  5. Adaptive Resource Allocation
  6. Mixed-precision Quantized Transformer
  7. Transition to Mixed-precision Quantization
  8. Resource-aware Mixed-precision Quantization
  9. Phase 1: Preparation
  10. Phase 2: Estimation
  11. Phase 3: Filtering
  12. Phase 4: Validation
  13. Preparation of Knowledge Database
  14. Resource Estimation and Filtering
  15. Experiments and Evaluation
  16. ...and 9 more sections

Figures (7)

  • Figure 1: The Architecture of the Transformer Model
  • Figure 2: Uniform (left) vs Mixed (right) 8-bit Linear Layer
  • Figure 3: Uniform (left) vs Mixed (right) 8-bit Addition Operation
  • Figure 4: Uniform (up) vs Mixed (down) 8-bit FFN Module
  • Figure 5: Workflow of Resource-aware Mixed-precision Quantization
  • ...and 2 more figures