A 1Mb mixed-precision quantized encoder for image classification and patch-based compression

TL;DR

The paper tackles the hardware-constraints of deploying DNNs at the edge by designing an ASIC-friendly mixed-precision encoder (NQE) capable of both image classification and patch-based compression. It introduces histogram-equidistributed quantization, HWMSB activations, and layer-shared BitShift Normalization (BSN) to streamline hardware implementation, while employing Group-wise Convolution and a compressed bottleneck to cut memory and compute. A dedicated decoder, PURENET, reconstructs full images from patch-based binary measurements, enabling end-to-end patch-based compression with minimal artifacts at 0.25 bpp. Empirical results on CIFAR-10 show CIFAR-10 accuracy of $87.48\%$ with only 1 Mb of on-chip memory, and full-frame DIV2K compression demonstrates favorable PSNR/MS-SSIM relative to standard patch-based methods, validating the hardware-software co-design approach for edge AI and image compression.

Abstract

Even if Application-Specific Integrated Circuits (ASIC) have proven to be a relevant choice for integrating inference at the edge, they are often limited in terms of applicability. In this paper, we demonstrate that an ASIC neural network accelerator dedicated to image processing can be applied to multiple tasks of different levels: image classification and compression, while requiring a very limited hardware. The key component is a reconfigurable, mixed-precision (3b/2b/1b) encoder that takes advantage of proper weight and activation quantizations combined with convolutional layer structural pruning to lower hardware-related constraints (memory and computing). We introduce an automatic adaptation of linear symmetric quantizer scaling factors to perform quantized levels equalization, aiming at stabilizing quinary and ternary weights training. In addition, a proposed layer-shared Bit-Shift Normalization significantly simplifies the implementation of the hardware-expensive Batch Normalization. For a specific configuration in which the encoder design only requires 1Mb, the classification accuracy reaches 87.5% on CIFAR-10. Besides, we also show that this quantized encoder can be used to compress image patch-by-patch while the reconstruction can performed remotely, by a dedicated full-frame decoder. This solution typically enables an end-to-end compression almost without any block artifacts, outperforming patch-based state-of-the-art techniques employing a patch-constant bitrate.

Figures (11)

• Figure 1: The joint framework for both image classification and image embedded compression and remote decompression.
• Figure 2: Schematic description of our framework involving neural network topology parts.
• Figure 3: Topology of the Nonlinear Quantized Encoder (NQE) + Classifier. F in red stands for the hyperparameter corresponding to the size scale of the feature map (i.e. the number of the feature map of the first convolution module). GC stands for Group-wise Convolution of 4 groups.
• Figure 4: Symmetric linear quantization with histogram bin equidistribution. The ternary quantization is represented with red dotted curve and red quantiles ${\color{red}q_i}$. The quinary quantization is represented with blue solid curve and blue quantile ${\color{blue(ncs)}q_j}$. For simplicity, we choose the same $\Delta$ for both two quantizations, but in practice, they usually have a different estimated $\Delta$.
• Figure 5: Example of Group-wise convolution with a 4-channel input divided into 2 groups and 8-channel output. The intermediate values are also divided into two groups, and each convolution is performed with a kernel that takes only two input channels from the corresponding group. These output channels are then structurally shuffled.