An Overview of Arithmetic Adaptations for Inference of Convolutional Neural Networks on Re-configurable Hardware

TL;DR

This work tackles efficient CNN inference on re-configurable hardware by targeting TinyYOLOv3 on an Artix-7 FPGA and proposing a hardware-aware adaptation workflow. It combines batch normalization fusion, filter pruning, and fixed-point quantization to reduce computational load, memory, and control complexity while maintaining detection performance; mathematical formulations and practical routines are provided for BN fusion ($W_{bn}=\frac{\gamma}{\sqrt{{\sigma^2}+\epsilon}}W$, $b_{bn}=\frac{\gamma}{\sqrt{{\sigma^2}+\epsilon}}(b-\mu)+\beta$) and pruning metrics (Frobenius norm and sparsity) with a MAP-driven pruning loop. The results demonstrate meaningful reductions in parameters and FLOPs (e.g., ~$23.1\%$–$27.7\%$ parameter reductions and ~$13.3\%$–$15.7\%$ FLOP reductions after BN fusion and pruning) and show that a quantized network can maintain acceptable accuracy, quantified via layer-wise MSE between floating-point and integer representations and final detection scores. These contributions enable practical, real-time inference of compact detectors on FPGA hardware, informing hardware-aware CNN design and deployment in embedded vision systems, with future work aimed at deeper quantization, weight sharing, and architecture-level adaptations like depthwise separable convolutions and binary networks.

Abstract

Convolutional Neural Networks (CNNs) have gained high popularity as a tool for computer vision tasks and for that reason are used in various applications. There are many different concepts, like single shot detectors, that have been published for detecting objects in images or video streams. However, CNNs suffer from disadvantages regarding the deployment on embedded platforms such as re-configurable hardware like Field Programmable Gate Arrays (FPGAs). Due to the high computational intensity, memory requirements and arithmetic conditions, a variety of strategies for running CNNs on FPGAs have been developed. The following methods showcase our best practice approaches for a TinyYOLOv3 detector network on a XILINX Artix-7 FPGA using techniques like fusion of batch normalization, filter pruning and post training network quantization.

Figures (6)

• Figure 1: Example of a general convolutional layer (abbreviated with conv) with its subsequent batchnorm, activation and pooling sub layer and their output descriptions.
• Figure 2: Required flop (y-axis, logarithmic) for all convolutional layer with their batchnorm sub layer (x-axis) for the TinyYOLOv3 architecture.
• Figure 3: Pruning result showing for the TinyYOLOv3 network example comparing fused model (fused) and pruned model with Frobenius Norm (fro) and Filter Sparsity (spars) as metric. Top of bar: reduction percentage or reference to it. Left: total parameter count. Right: filter count of the network.
• Figure 4: Percentage of parameters for each layer. Per layer and cumulative.
• Figure 5: flop comparison (y-axis) for the TinyYOLOv3 network example. Reduction percentages (top of bar) using the "unfused" original model (orig) as reference.