Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Continuous-Flow Data-Rate-Aware CNN Inference on FPGA

Tobias Habermann, Michael Mecik, Zhenyu Wang, César David Vera, Martin Kumm, Mario Garrido

TL;DR

The paper addresses data-rate bottlenecks in FPGA-based CNN inference caused by pooling and strided layers. It proposes a continuous-flow, data-rate-aware architecture with interleaving and reconfigurable units to sustain high hardware utilization and throughput. Key contributions include a layer-wise data-flow analysis (KPU/PPU/FCU), a complexity framework, an automated code generator, and synthesis results showing substantial resource savings while enabling MobileNet on a single FPGA with 8-bit quantization. By bridging stream and unrolled designs, the approach delivers scalable, high-throughput CNN inference suitable for real-time applications on reconfigurable hardware.

Abstract

Among hardware accelerators for deep-learning inference, data flow implementations offer low latency and high throughput capabilities. In these architectures, each neuron is mapped to a dedicated hardware unit, making them well-suited for field-programmable gate array (FPGA) implementation. Previous unrolled implementations mostly focus on fully connected networks because of their simplicity, although it is well known that convolutional neural networks (CNNs) require fewer computations for the same accuracy. When observing the data flow in CNNs, pooling layers and convolutional layers with a stride larger than one, the number of data at their output is reduced with respect to their input. This data reduction strongly affects the data rate in a fully parallel implementation, making hardware units heavily underutilized unless it is handled properly. This work addresses this issue by analyzing the data flow of CNNs and presents a novel approach to designing data-rate-aware, continuous-flow CNN architectures. The proposed approach ensures a high hardware utilization close to 100% by interleaving low data rate signals and sharing hardware units, as well as using the right parallelization to achieve the throughput of a fully parallel implementation. The results show that a significant amount of the arithmetic logic can be saved, which allows implementing complex CNNs like MobileNet on a single FPGA with high throughput.

Continuous-Flow Data-Rate-Aware CNN Inference on FPGA

TL;DR

The paper addresses data-rate bottlenecks in FPGA-based CNN inference caused by pooling and strided layers. It proposes a continuous-flow, data-rate-aware architecture with interleaving and reconfigurable units to sustain high hardware utilization and throughput. Key contributions include a layer-wise data-flow analysis (KPU/PPU/FCU), a complexity framework, an automated code generator, and synthesis results showing substantial resource savings while enabling MobileNet on a single FPGA with 8-bit quantization. By bridging stream and unrolled designs, the approach delivers scalable, high-throughput CNN inference suitable for real-time applications on reconfigurable hardware.

Abstract

Among hardware accelerators for deep-learning inference, data flow implementations offer low latency and high throughput capabilities. In these architectures, each neuron is mapped to a dedicated hardware unit, making them well-suited for field-programmable gate array (FPGA) implementation. Previous unrolled implementations mostly focus on fully connected networks because of their simplicity, although it is well known that convolutional neural networks (CNNs) require fewer computations for the same accuracy. When observing the data flow in CNNs, pooling layers and convolutional layers with a stride larger than one, the number of data at their output is reduced with respect to their input. This data reduction strongly affects the data rate in a fully parallel implementation, making hardware units heavily underutilized unless it is handled properly. This work addresses this issue by analyzing the data flow of CNNs and presents a novel approach to designing data-rate-aware, continuous-flow CNN architectures. The proposed approach ensures a high hardware utilization close to 100% by interleaving low data rate signals and sharing hardware units, as well as using the right parallelization to achieve the throughput of a fully parallel implementation. The results show that a significant amount of the arithmetic logic can be saved, which allows implementing complex CNNs like MobileNet on a single FPGA with high throughput.
Paper Structure (29 sections, 37 equations, 13 figures, 10 tables)

This paper contains 29 sections, 37 equations, 13 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Convolutional Neural Networks
  4. Convolutional Layers
  5. Pooling Layers
  6. Fully Connected Layers
  7. Continuous Flow in CNN Layers
  8. Continuous Flow in Convolutional Layers
  9. Continuous Flow with Padding in Convolutional Layers
  10. Continuous Flow With Stride In Convolutional Layers
  11. Continuous Flow in Pooling Layers
  12. Continuous Flow in Fully Connected Layers
  13. Building an Entire Continuous-Flow CNN
  14. Running Example
  15. Data Rates at the CNN Layers
  16. ...and 14 more sections

Figures (13)

  • Figure 1: The typical structure of a CNN.
  • Figure 2: KPU to calculate a $3 \times 3$ convolutional kernel.
  • Figure 3: Diagrams to analyze continuous flow with and without padding.
  • Figure 4: A KPU with added zero-padding functionality.
  • Figure 5: An example for a $2 \times 2$ max-pooling operation.
  • ...and 8 more figures