InTAR: Inter-Task Auto-Reconfigurable Accelerator Design for High Data Volume Variation in DNNs

TL;DR

The Inter-Task Auto- Reconfigurable Accelerator (InTAR) is introduced, a novel accelerator design methodology for HDV applications on FPGAs that combines the high computational efficiency of sequential execution with the reduced off-chip memory overhead of dataflow execution.

Abstract

The rise of deep neural networks (DNNs) has driven an increased demand for computing power and memory. Modern DNNs exhibit high data volume variation (HDV) across tasks, which poses challenges for FPGA acceleration: conventional accelerators rely on fixed execution patterns (dataflow or sequential) that can lead to pipeline stalls or necessitate frequent off-chip memory accesses. To address these challenges, we introduce the Inter-Task Auto-Reconfigurable Accelerator (InTAR), a novel accelerator design methodology for HDV applications on FPGAs. InTAR combines the high computational efficiency of sequential execution with the reduced off-chip memory overhead of dataflow execution. It switches execution patterns automatically with a static schedule determined before circuit design based on resource constraints and problem sizes. Unlike previous reconfigurable accelerators, InTAR encodes reconfiguration schedules during circuit design, allowing model-specific optimizations that allocate only the necessary logic and interconnects. Thus, InTAR achieves a high clock frequency with fewer resources and low reconfiguration time. Furthermore, InTAR supports high-level tools such as HLS for fast design generation. We implement a set of multi-task HDV DNN kernels using InTAR. Compared with dataflow and sequential accelerators, InTAR exhibits $\mathbf{1.8\times}$ and $\mathbf{7.1 \times}$ speedups correspondingly. Moreover, we extend InTAR to GPT-2 medium as a more complex example, which is $\mathbf{3.65 \sim 39.14\times}$ faster and a $\mathbf{1.72 \sim 10.44\times}$ more DSP efficient than SoTA accelerators (Allo and DFX) on FPGAs. Additionally, this design demonstrates $\mathbf{1.66 \sim 7.17\times}$ better power efficiency than GPUs. Code: https://github.com/OswaldHe/InTAR

Figures (8)

• Figure 1: Example of mapping computations of attention and linear projection of value matrix to the dataflow, sequential, hybrid accelerators, and InTAR. The sequence length is 256, the hidden dimension is 1024, and the weights are $1024\times 1024$ matrices. Depending on the scheduling of the rest of the computations in the entire model, InTAR can choose between task-parallel (left) and sequential modes (right) for better locality. Both will have the same latency.
• Figure 2: Left: Architecture template of InTAR. Compute cores (CC) compute linear operations (e.g., GEMM, ConvNet), and SFUs compute non-linear operations (e.g., softmax, GeLU). Each CC contains a scratchpad memory, a reconfigurable MAC unit array, a reduction unit, and a data movement control unit. Dashed lines indicate the candidates for connection between CCs and SFUs. Right: example architectures for each execution mode within the template, with $n,m=2$. For a schedule, we will compose the architectures of the required modes to keep only the necessary interconnects and logic.
• Figure 3: Example of data movement reconfiguration. Left: In different stages, PE 3 either read from PE 1 and write to on-chip buffer buf 1, or read from buf 2 and write to PE 2. Right: the corresponding HLS code.
• Figure 4: Example of compute reconfiguration. Left: The reconfigurable PEA consumes different buffers at each stage and supports mixed precision of b-bit and c-bit calculations. Right: the corresponding HLS code.
• Figure 5: Example of control reconfigurations in HLS. Left: loop-bound control. Right: data-dependent conditional dataflow.