TrIM, Triangular Input Movement Systolic Array for Convolutional Neural Networks: Architecture and Hardware Implementation

TL;DR

This work tackles the memory-bandwidth bottleneck in CNN accelerators by introducing Triangular Input Movement (TrIM), a dataflow that reduces data movement and memory accesses. It presents a hierarchical TrIM hardware architecture (Slice, Core, Engine) with Reconfigurable Shift Register Buffers to support variable ifmap sizes, and demonstrates an FPGA implementation for VGG-16 and AlexNet. The TrIM design achieves up to $453.6$ GOPs/s peak throughput on a $1512$-PE FPGA engine and reduces memory accesses by up to about $3\times$ compared with a state-of-the-art RS-based accelerator, with energy efficiency gains up to $11.9\times$. Design-space analysis shows scalable performance across PN and PM under realistic I/O bandwidth constraints, underscoring TrIM’s practical impact for energy-efficient CNN inference on reconfigurable hardware.

Abstract

Modern hardware architectures for Convolutional Neural Networks (CNNs), other than targeting high performance, aim at dissipating limited energy. Reducing the data movement cost between the computing cores and the memory is a way to mitigate the energy consumption. Systolic arrays are suitable architectures to achieve this objective: they use multiple processing elements that communicate each other to maximize data utilization, based on proper dataflows like the weight stationary and row stationary. Motivated by this, we have proposed TrIM, an innovative dataflow based on a triangular movement of inputs, and capable to reduce the number of memory accesses by one order of magnitude when compared to state-of-the-art systolic arrays. In this paper, we present a TrIM-based hardware architecture for CNNs. As a showcase, the accelerator is implemented onto a Field Programmable Gate Array (FPGA) to execute the VGG-16 and AlexNet CNNs. The architecture achieves a peak throughput of 453.6 Giga Operations per Second, outperforming a state-of-the-art row stationary systolic array up to ~3x in terms of memory accesses, and being up to ~11.9x more energy-efficient than other FPGA accelerators.

Figures (7)

• Figure 1: VGG-16 CNN: Memory requirements and operations for each Convolutional Layer (CL). These metrics refer to the inference step at which a generic 224$\times$224 RGB image must be subjected. The memory requirements, expressed in MBytes, are represented by bars. For each bar, both the ifmaps and weights memory are reported. The number of operations, in billions, are highlighted by the dashed line.
• Figure 2: Example of Convolutional Layer (CL) using the TrIM-based architecture: (a) The CL processes $M=4$ ifmaps and $N=2$ filters, each consisting of $M=2$ kernels. Each kernel accommodates $K \times K = 3 \times 3$ weights. As a result, $N=2$ ofmaps are generated; (b) example of processing of the first two ifmaps and the first two kernels of each filter. The TrIM Engine consists of two TrIM Cores, each having two TrIM Slices. The first TrIM Core processes ifmaps 0 and 1, as well as the first filter (kernels 00 and 01). The second TrIM Core processes ifmaps 0 and 1, and the second filter (kernels 10 and 11). Both TrIM Cores generate provisional ofmaps resulting as accumulations of the psums coming from their respective slices. Finally, the TrIM Engine is also responsible to accumulate over time the provisional ofmaps coming from the cores. In the specific case, the TrIM Engine accumulates the results from ifmaps 0 and 1, with the results from ifamps 2 and 3 in the subsequent iteration (not shown in figure).
• Figure 3: The TrIM Slice. This consists of $K$ rows of $K$ Processing Elements (PEs) each, as well as $K-1$ Reconfigurable Shift Register Buffers (RSRBs) to facilitate the diagonal movement of inputs. For each PE, the arrows with different colors refer to a specific data/movement: the blue arrows indicate the external inputs supplied to the array; the purple arrows refers to the weights first supplied externally and, then, propagated from top to down; (c) the red arrows indicate the right-to-left movement of inputs; (d) the brown arrows indicate the diagonal movement of inputs from RSRBs to PEs; (e) the green arrows refer to the psums accumulation from top to down. A detail of the generic PE is also reported: it consists of four registers (in light-blue), two multiplexer to route the correct input, and the Multiply-Accumulation (MAC) unit. Finally, it is worth underlining that the outputs from the RSRBs can be seen as buses of $K$ inputs, which in turn feed as many PEs placed on the top.
• Figure 4: The Reconfigurable Shift Register Buffer (RSRB). This consists of multiple Sub-Buffers (SBs), each having $L_{sb}$ shift registers, with $sb$ iterating over the number of SBs. $L_{sb}$ can be generic or customized parameters. Each SB provides the leftmost $K$ inputs to a multiplexer that, according to a given selection signal, provides the target set of $K$ inputs to the PEs placed in the top Row, in order to meet the diagonal movement of inputs.
• Figure 5: The TrIM Core. It accommodates $P_M$ TrIM Slices, with $P_M$ being the number of parallel ifmaps processed by the accelerator. In addition, the TrIM Core spatially accumulates the outputs coming from the slices through an adder tree having a number of stages proportional to the logarithm in base 2 of $P_M$. The output is thus registered before being provided to further processing modules.