A Flexible Precision Scaling Deep Neural Network Accelerator with Efficient Weight Combination

TL;DR

This work tackles edge inference with mixed-precision deep neural networks by introducing a precision-scalable accelerator that supports 2 to 8 bit weights and activations. It employs weight decomposition via two load modes, a weight-stationary bit-serial MAC with systolic dataflow, and a carry-save adder tree to efficiently sum partial results. Experimental results in 28 nm show strong energy efficiency gains (up to tens to hundreds of TOPS per watt) and higher hardware utilization compared to prior work, including improved peak throughput across bitwidths and meaningful power reductions on MobileNetV2. The design offers a practical pathway to efficient edge inference with fine-grained precision control and low reconfiguration overhead.

Abstract

Deploying mixed-precision neural networks on edge devices is friendly to hardware resources and power consumption. To support fully mixed-precision neural network inference, it is necessary to design flexible hardware accelerators for continuous varying precision operations. However, the previous works have issues on hardware utilization and overhead of reconfigurable logic. In this paper, we propose an efficient accelerator for 2~8-bit precision scaling with serial activation input and parallel weight preloaded. First, we set two loading modes for the weight operands and decompose the weight into the corresponding bitwidths, which extends the weight precision support efficiently. Then, to improve hardware utilization of low-precision operations, we design the architecture that performs bit-serial MAC operation with systolic dataflow, and the partial sums are combined spatially. Furthermore, we designed an efficient carry save adder tree supporting both signed and unsigned number summation across rows. The experiment result shows that the proposed accelerator, synthesized with TSMC 28nm CMOS technology, achieves peak throughput of 4.09TOPS and peak energy efficiency of 68.94TOPS/W at 2/2-bit operations.

Figures (8)

• Figure 1: Architecture of (a) yang2020bitsystolic and (b) chi202116.
• Figure 2: The overall architecture.
• Figure 3: (a) Schematic of the multiplier. (b) Load weight in 2-bit mode. (c) Decompose 5-bit weight with 3-bit and 2-bit mode.
• Figure 4: Independent shift-add path for summation in 6/7-bit weight operation.
• Figure 5: Schematic of configurable shift-add logic.