USEFUSE: Uniform Stride for Enhanced Performance in Fused Layer Architecture of Deep Neural Networks

TL;DR

This work addresses the challenge of deploying deep CNNs on resource-constrained edge devices by introducing Sum-of-Products units built on low-latency online MSDF left-to-right arithmetic and fusing multiple convolution layers to dramatically reduce off-chip memory traffic. A key contribution is the END mechanism, which terminates ineffectual convolutions after ReLU without accuracy loss, combined with a uniform fusion-pyramid tiling strategy that ensures consistent tile movement. Two accelerator designs, DS-1 (spatial) and DS-2 (temporal), are proposed to cover both minimal latency and low-resource scenarios, achieving large gains in operational intensity ($8.2\times$, $17.8\times$, $279.4\times$ for LeNet-5, AlexNet, VGG-16 respectively) and energy savings around $46.8\%$–$48.5\%$, alongside significant BRAM reductions on large networks. The results demonstrate substantial throughput and latency improvements over prior CNN accelerators, underscoring the practical impact of uniform stride fusion and online arithmetic for edge-AI workloads, with clear potential for extension to transformer-based models and other activations.

Abstract

Convolutional Neural Networks (CNNs) are crucial in various applications, but their deployment on resource-constrained edge devices poses challenges. This study presents the Sum-of-Products (SOP) units for convolution, which utilize low-latency left-to-right bit-serial arithmetic to minimize response time and enhance overall performance. The study proposes a methodology for fusing multiple convolution layers to reduce off-chip memory communication and increase overall performance. An effective mechanism detects and skips inefficient convolutions after ReLU layers, minimizing power consumption without compromising accuracy. Furthermore, efficient tile movement guarantees uniform access to the fusion pyramid. An analysis demonstrates the utile stride strategy improves operational intensity. Two designs cater to varied demands: one focuses on minimal response time for mission-critical applications, and another focuses on resource-constrained devices with comparable latency. This approach notably reduced redundant computations, improving the efficiency of CNN deployment on edge devices.

Figures (14)

• Figure 1: A general CNN architecture.
• Figure 2: Proposed layer fusion accelerator design pipeline
• Figure 3: General layer fusion scheme
• Figure 4: Overall Architecture. The solid black arrows represent the input, output, and control connections, while the dotted green arrows represent the filter/weight data.
• Figure 5: Tile/Pyramid Level Design