2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution

TL;DR

2DQuant introduces a dual-stage post-training quantization framework for Transformer-based image super-resolution, addressing the degradation typically caused by low-bit quantization. The method first uses Distribution-Oriented Bound Initialization (DOBI) to rapidly find coarse, distribution-aware clipping bounds, then applies Distillation Quantization Calibration (DQC) to fine-tune bounds via teacher-student distillation, aligning the quantized model with the full-precision counterpart. By exploiting the distinct weight and activation distributions in SwinIR, 2DQuant achieves state-of-the-art PTQ results, enabling 4-, 3-, and 2-bit quantization with substantial compression (up to 3.60×) and speedup (up to 5.08×) while preserving or even surpassing FP performance in challenging cases. The approach demonstrates strong practicality for edge deployment and upgrades SR workflows with transformer-based architectures.

Abstract

Low-bit quantization has become widespread for compressing image super-resolution (SR) models for edge deployment, which allows advanced SR models to enjoy compact low-bit parameters and efficient integer/bitwise constructions for storage compression and inference acceleration, respectively. However, it is notorious that low-bit quantization degrades the accuracy of SR models compared to their full-precision (FP) counterparts. Despite several efforts to alleviate the degradation, the transformer-based SR model still suffers severe degradation due to its distinctive activation distribution. In this work, we present a dual-stage low-bit post-training quantization (PTQ) method for image super-resolution, namely 2DQuant, which achieves efficient and accurate SR under low-bit quantization. The proposed method first investigates the weight and activation and finds that the distribution is characterized by coexisting symmetry and asymmetry, long tails. Specifically, we propose Distribution-Oriented Bound Initialization (DOBI), using different searching strategies to search a coarse bound for quantizers. To obtain refined quantizer parameters, we further propose Distillation Quantization Calibration (DQC), which employs a distillation approach to make the quantized model learn from its FP counterpart. Through extensive experiments on different bits and scaling factors, the performance of DOBI can reach the state-of-the-art (SOTA) while after stage two, our method surpasses existing PTQ in both metrics and visual effects. 2DQuant gains an increase in PSNR as high as 4.52dB on Set5 (x2) compared with SOTA when quantized to 2-bit and enjoys a 3.60x compression ratio and 5.08x speedup ratio. The code and models will be available at https://github.com/Kai-Liu001/2DQuant.

Figures (13)

• Figure 1: Existing methods suffer from blurring artifacts.
• Figure 2: The overall pipeline of our proposed 2DQuant method. The whole pipeline contains two stages, optimizing the clipping bound from coarse to fine. In stage 1, we design DOBI to efficiently obtain the coarse bound. In stage 2, DQC is performed to finetune clipping bounds and guarantee the quantized model learns the full-precision (FP) model's feature and output information.
• Figure 3: Quantization scheme for SwinIR Transformer blocks. Fake quantization and INT arithmetic are performed in all compute-intensive operators including all linear layers and batch matmul. Lower bits such as 3 or even 2 are also permitted. Dropout of attention and projection is ignored
• Figure 4: The selected representative distribution of activations (Row 1) and weights (Row 2). The range of data is marked in the figure. All weights obey symmetric distribution. The attention map and the input of FC2 are asymmetric due to softmax function and GELU function.
• Figure 5: The bound percentile of DOBI and DQC.