Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FPGA-based Acceleration for Convolutional Neural Networks: A Comprehensive Review

Junye Jiang, Yaan Zhou, Yuanhao Gong, Haoxuan Yuan, Shuanglong Liu

TL;DR

This survey addresses the challenge of running large CNNs within resource-constrained environments by evaluating FPGA-based CNN accelerators. It synthesizes algorithmic optimization (pruning, quantization, distillation, layer fusion, FFT/Winograd) with hardware strategies (dataflow, tiling, unrolling, parallelism) and formal evaluation metrics that relate model performance to hardware efficiency. The authors introduce a dual-focus evaluation framework (model- and hardware-level metrics) and discuss toolflows, design-space exploration methods, and performance/resource modeling to guide FPGA implementations. The work highlights dynamic parallelism and co-design as central themes for achieving high compute and overall efficiency, especially in edge and embedded deployments, while outlining future directions in heterogeneous computing and AI-driven DSE for CNN accelerators.

Abstract

Convolutional Neural Networks (CNNs) are fundamental to deep learning, driving applications across various domains. However, their growing complexity has significantly increased computational demands, necessitating efficient hardware accelerators. Field-Programmable Gate Arrays (FPGAs) have emerged as a leading solution, offering reconfigurability, parallelism, and energy efficiency. This paper provides a comprehensive review of FPGA-based hardware accelerators specifically designed for CNNs. It presents and summarizes the performance evaluation framework grounded in existing studies and explores key optimization strategies, such as parallel computing, dataflow optimization, and hardware-software co-design. It also compares various FPGA architectures in terms of latency, throughput, compute efficiency, power consumption, and resource utilization. Finally, the paper highlights future challenges and opportunities, emphasizing the potential for continued innovation in this field.

FPGA-based Acceleration for Convolutional Neural Networks: A Comprehensive Review

TL;DR

This survey addresses the challenge of running large CNNs within resource-constrained environments by evaluating FPGA-based CNN accelerators. It synthesizes algorithmic optimization (pruning, quantization, distillation, layer fusion, FFT/Winograd) with hardware strategies (dataflow, tiling, unrolling, parallelism) and formal evaluation metrics that relate model performance to hardware efficiency. The authors introduce a dual-focus evaluation framework (model- and hardware-level metrics) and discuss toolflows, design-space exploration methods, and performance/resource modeling to guide FPGA implementations. The work highlights dynamic parallelism and co-design as central themes for achieving high compute and overall efficiency, especially in edge and embedded deployments, while outlining future directions in heterogeneous computing and AI-driven DSE for CNN accelerators.

Abstract

Convolutional Neural Networks (CNNs) are fundamental to deep learning, driving applications across various domains. However, their growing complexity has significantly increased computational demands, necessitating efficient hardware accelerators. Field-Programmable Gate Arrays (FPGAs) have emerged as a leading solution, offering reconfigurability, parallelism, and energy efficiency. This paper provides a comprehensive review of FPGA-based hardware accelerators specifically designed for CNNs. It presents and summarizes the performance evaluation framework grounded in existing studies and explores key optimization strategies, such as parallel computing, dataflow optimization, and hardware-software co-design. It also compares various FPGA architectures in terms of latency, throughput, compute efficiency, power consumption, and resource utilization. Finally, the paper highlights future challenges and opportunities, emphasizing the potential for continued innovation in this field.
Paper Structure (45 sections, 11 equations, 3 figures, 6 tables, 1 algorithm)

This paper contains 45 sections, 11 equations, 3 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. CNN Acceleration Platforms
  3. Hardware Platforms
  4. Further Comparison of FPGAs and GPUs
  5. Discussion
  6. Evaluation Framework
  7. Model Performance Evaluation
  8. Hardware Performance Evaluation
  9. Discussion on Evaluation Framework
  10. CNN Acceleration Methods
  11. Pruning and Quantization
  12. Pruning
  13. Quantization
  14. Model Structure Compression
  15. Lightweight Networks
  16. ...and 30 more sections

Figures (3)

  • Figure 1: An illustration depicting the four levels of parallelism in convolution computation across the filter ($P_f$), channel ($P_c$), pixel ($P_v$), and kernel ($P_k$) dimensions.
  • Figure 2: Energy efficiency comparison across CNN accelerator architectures. The left vertical axis represents energy efficiency, while the right vertical axis shows power consumption.
  • Figure 3: An illustrative comparison of the hardware performance of existing CNN accelerators in terms of power consumption, latency, and throughput. The size of the bubbles corresponds to latency, with larger bubbles indicating higher latency and smaller ones indicating lower latency. The horizontal axis represents power consumption, increasing from left to right, while the vertical axis represents throughput, increasing from bottom to top.