Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAs

Dingyi Dai, Yichi Zhang, Jiahao Zhang, Zhanqiu Hu, Yaohui Cai, Qi Sun, Zhiru Zhang

TL;DR

This work tackles the misalignment between floating-point training and fixed-point FPGA inference by introducing QFX, a differentiable fixed-point quantization framework that learns per-layer binary-point positions during training and supports a multiplier-free K-hot quantization to reduce DSP usage. Implemented as a PyTorch-based emulation that mirrors FPGA-HLS behavior, QFX enables end-to-end quantization-aware training whose deployed models match fixed-point hardware results. Empirical results demonstrate that QFX surpasses post-training quantization at low bitwidth on CIFAR-10 and ImageNet and can achieve DSP-free operation in BNN accelerators on embedded FPGAs. The approach offers practical, deployment-ready fixed-point DNNs with improved accuracy and reduced hardware resources, with open-source release planned.

Abstract

Quantization is a crucial technique for deploying deep learning models on resource-constrained devices, such as embedded FPGAs. Prior efforts mostly focus on quantizing matrix multiplications, leaving other layers like BatchNorm or shortcuts in floating-point form, even though fixed-point arithmetic is more efficient on FPGAs. A common practice is to fine-tune a pre-trained model to fixed-point for FPGA deployment, but potentially degrading accuracy. This work presents QFX, a novel trainable fixed-point quantization approach that automatically learns the binary-point position during model training. Additionally, we introduce a multiplier-free quantization strategy within QFX to minimize DSP usage. QFX is implemented as a PyTorch-based library that efficiently emulates fixed-point arithmetic, supported by FPGA HLS, in a differentiable manner during backpropagation. With minimal effort, models trained with QFX can readily be deployed through HLS, producing the same numerical results as their software counterparts. Our evaluation shows that compared to post-training quantization, QFX can quantize models trained with element-wise layers quantized to fewer bits and achieve higher accuracy on both CIFAR-10 and ImageNet datasets. We further demonstrate the efficacy of multiplier-free quantization using a state-of-the-art binarized neural network accelerator designed for an embedded FPGA (AMD Xilinx Ultra96 v2). We plan to release QFX in open-source format.

Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAs

TL;DR

This work tackles the misalignment between floating-point training and fixed-point FPGA inference by introducing QFX, a differentiable fixed-point quantization framework that learns per-layer binary-point positions during training and supports a multiplier-free K-hot quantization to reduce DSP usage. Implemented as a PyTorch-based emulation that mirrors FPGA-HLS behavior, QFX enables end-to-end quantization-aware training whose deployed models match fixed-point hardware results. Empirical results demonstrate that QFX surpasses post-training quantization at low bitwidth on CIFAR-10 and ImageNet and can achieve DSP-free operation in BNN accelerators on embedded FPGAs. The approach offers practical, deployment-ready fixed-point DNNs with improved accuracy and reduced hardware resources, with open-source release planned.

Abstract

Quantization is a crucial technique for deploying deep learning models on resource-constrained devices, such as embedded FPGAs. Prior efforts mostly focus on quantizing matrix multiplications, leaving other layers like BatchNorm or shortcuts in floating-point form, even though fixed-point arithmetic is more efficient on FPGAs. A common practice is to fine-tune a pre-trained model to fixed-point for FPGA deployment, but potentially degrading accuracy. This work presents QFX, a novel trainable fixed-point quantization approach that automatically learns the binary-point position during model training. Additionally, we introduce a multiplier-free quantization strategy within QFX to minimize DSP usage. QFX is implemented as a PyTorch-based library that efficiently emulates fixed-point arithmetic, supported by FPGA HLS, in a differentiable manner during backpropagation. With minimal effort, models trained with QFX can readily be deployed through HLS, producing the same numerical results as their software counterparts. Our evaluation shows that compared to post-training quantization, QFX can quantize models trained with element-wise layers quantized to fewer bits and achieve higher accuracy on both CIFAR-10 and ImageNet datasets. We further demonstrate the efficacy of multiplier-free quantization using a state-of-the-art binarized neural network accelerator designed for an embedded FPGA (AMD Xilinx Ultra96 v2). We plan to release QFX in open-source format.
Paper Structure (16 sections, 12 equations, 3 figures, 6 tables, 3 algorithms)

This paper contains 16 sections, 12 equations, 3 figures, 6 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Integer Quantization
  4. Fixed-point Quantization
  5. Methodology
  6. QFX Implementation
  7. Learning Binary Point Position
  8. K-hot Fixed-point Quantization
  9. Evaluation
  10. Evaluation of Model Accuracy
  11. Quantization configurations.
  12. Training details.
  13. Accuracy Results.
  14. FPGA Implementation
  15. Related Work
  16. ...and 1 more sections

Figures (3)

  • Figure 1: QFX design flow. Element-wise operations are cast to fixed-point at training time without the "quantization" and "dequantization" overhead. After QAT, exactly the same model will be deployed on the target hardware, with the casting function replaced by hardened circuits.
  • Figure 2: Resource usage on an AMD Xilinx Ultra96 v2 FPGA.
  • Figure 3: DSP breakdown w/ different quantization methods.