HASS: Hardware-Aware Sparsity Search for Dataflow DNN Accelerator

TL;DR

HASS introduces hardware-aware sparsity search to co-optimize unstructured weight and activation sparsity for dataflow DNN accelerators. By integrating hardware metrics into a joint pruning and design-space exploration workflow, it performs a multi-objective optimization (via TPE) over layer-wise pruning thresholds and accelerator configurations. The approach achieves 1.3×–4.2× efficiency gains over prior sparse designs across multiple models, with MobileNetV3 reaching 4895 images/s on FPGA. This hardware-software co-design enables better utilization of on-chip resources and scales sparsity benefits to dataflow pipelines, offering practical gains for energy-efficient DNN inference. The work is open-source, facilitating further adoption and extension to diverse hardware platforms and pruning strategies.

Abstract

Deep Neural Networks (DNNs) excel in learning hierarchical representations from raw data, such as images, audio, and text. To compute these DNN models with high performance and energy efficiency, these models are usually deployed onto customized hardware accelerators. Among various accelerator designs, dataflow architecture has shown promising performance due to its layer-pipelined structure and its scalability in data parallelism. Exploiting weights and activations sparsity can further enhance memory storage and computation efficiency. However, existing approaches focus on exploiting sparsity in non-dataflow accelerators, which cannot be applied onto dataflow accelerators because of the large hardware design space introduced. As such, this could miss opportunities to find an optimal combination of sparsity features and hardware designs. In this paper, we propose a novel approach to exploit unstructured weights and activations sparsity for dataflow accelerators, using software and hardware co-optimization. We propose a Hardware-Aware Sparsity Search (HASS) to systematically determine an efficient sparsity solution for dataflow accelerators. Over a set of models, we achieve an efficiency improvement ranging from 1.3$\times$ to 4.2$\times$ compared to existing sparse designs, which are either non-dataflow or non-hardware-aware. Particularly, the throughput of MobileNetV3 can be optimized to 4895 images per second. HASS is open-source: \url{https://github.com/Yu-Zhewen/HASS}

Figures (6)

• Figure 1: HASS explores the optimal trade-off between classification accuracy and operation density. We compare HASS with other existing sparse MobileNetV2 implementations.
• Figure 2: An overview of our approach. When designing a dataflow accelerator, traditional pruning steps are separate from the hardware synthesis steps, which restricts software and hardware co-optimization. Our flow explores both software and hardware optimization concurrently, opening up opportunities to find the optimal design.
• Figure 3: Architecture of the sparse dataflow accelerator. The computation of layers is pipelined where inside each layer, there are multiple Sparse Vector Dot Product Engines (SPE) operating in parallel. Inside the SPE, clipping modules zero out any weight or activation that falls below a configurable threshold. Zero-filtering then detects these zeros, while the remaining non-zeros are dispatched to the MACs (implemented with DSPs) via an arbiter. We also use a dedicated counter to track the number of skipped zeros and manage accumulation result emissions.
• Figure 4: DSE results of a specific sparse ResNet-18 workload with 16 $3\times3$ convolutional layers. The allocation of MAC per SPE mainly depends on the per-layer sparsity statistic. A higher sparsity leads to a smaller MAC per SPE. However, the increase of the layer index leads to an increasing number of convolutional filters, so as the number parallel SPEs to match the rate between layers.
• Figure 5: Comparison between software metrics-only sparsity search and the proposed hardware-aware sparsity search for ResNet-18. We use 96 iteration steps for both approaches.