Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TAPA-CS: Enabling Scalable Accelerator Design on Distributed HBM-FPGAs

Neha Prakriya, Yuze Chi, Suhail Basalama, Linghao Song, Jason Cong

TL;DR

TAPA-CS addresses the lack of scalable programming models for cloud-scale, network-connected FPGAs by introducing a task-parallel dataflow framework that automatically partitions and compiles large designs across multiple FPGAs. It couples ILP-based inter- and intra-FPGA floorplanning with interconnect pipelining to maximize frequency while balancing resources, leveraging latency-insensitive dataflow for flexible module placement. The approach is validated on 2–4 FPGAs with benchmarks including Stencil, PageRank, KNN, and CNN, achieving average throughput improvements of 2.1x, 3.2x, and 4.4x for 2, 3, and 4 FPGAs, along with 11–116% frequency gains over single-FPGA baselines. The results demonstrate scalable, high-throughput accelerator design on cloud FPGAs with moderate overhead and broad applicability, with plans to open-source the tool for broader adoption.

Abstract

Despite the increasing adoption of Field-Programmable Gate Arrays (FPGAs) in compute clouds, there remains a significant gap in programming tools and abstractions which can leverage network-connected, cloud-scale, multi-die FPGAs to generate accelerators with high frequency and throughput. To this end, we propose TAPA-CS, a task-parallel dataflow programming framework which automatically partitions and compiles a large design across a cluster of FPGAs with no additional user effort while achieving high frequency and throughput. TAPA-CS has three main contributions. First, it is an open-source framework which allows users to leverage virtually "unlimited" accelerator fabric, high-bandwidth memory (HBM), and on-chip memory, by abstracting away the underlying hardware. This reduces the user's programming burden to a logical one, enabling software developers and researchers with limited FPGA domain knowledge to deploy larger designs than possible earlier. Second, given as input a large design, TAPA-CS automatically partitions the design to map to multiple FPGAs, while ensuring congestion control, resource balancing, and overlapping of communication and computation. Third, TAPA-CS couples coarse-grained floorplanning with automated interconnect pipelining at the inter- and intra-FPGA levels to ensure high frequency. We have tested TAPA-CS on our multi-FPGA testbed where the FPGAs communicate through a high-speed 100Gbps Ethernet infrastructure. We have evaluated the performance and scalability of our tool on designs, including systolic-array based convolutional neural networks (CNNs), graph processing workloads such as page rank, stencil applications like the Dilate kernel, and K-nearest neighbors (KNN). TAPA-CS has the potential to accelerate development of increasingly complex and large designs on the low power and reconfigurable FPGAs.

TAPA-CS: Enabling Scalable Accelerator Design on Distributed HBM-FPGAs

TL;DR

TAPA-CS addresses the lack of scalable programming models for cloud-scale, network-connected FPGAs by introducing a task-parallel dataflow framework that automatically partitions and compiles large designs across multiple FPGAs. It couples ILP-based inter- and intra-FPGA floorplanning with interconnect pipelining to maximize frequency while balancing resources, leveraging latency-insensitive dataflow for flexible module placement. The approach is validated on 2–4 FPGAs with benchmarks including Stencil, PageRank, KNN, and CNN, achieving average throughput improvements of 2.1x, 3.2x, and 4.4x for 2, 3, and 4 FPGAs, along with 11–116% frequency gains over single-FPGA baselines. The results demonstrate scalable, high-throughput accelerator design on cloud FPGAs with moderate overhead and broad applicability, with plans to open-source the tool for broader adoption.

Abstract

Despite the increasing adoption of Field-Programmable Gate Arrays (FPGAs) in compute clouds, there remains a significant gap in programming tools and abstractions which can leverage network-connected, cloud-scale, multi-die FPGAs to generate accelerators with high frequency and throughput. To this end, we propose TAPA-CS, a task-parallel dataflow programming framework which automatically partitions and compiles a large design across a cluster of FPGAs with no additional user effort while achieving high frequency and throughput. TAPA-CS has three main contributions. First, it is an open-source framework which allows users to leverage virtually "unlimited" accelerator fabric, high-bandwidth memory (HBM), and on-chip memory, by abstracting away the underlying hardware. This reduces the user's programming burden to a logical one, enabling software developers and researchers with limited FPGA domain knowledge to deploy larger designs than possible earlier. Second, given as input a large design, TAPA-CS automatically partitions the design to map to multiple FPGAs, while ensuring congestion control, resource balancing, and overlapping of communication and computation. Third, TAPA-CS couples coarse-grained floorplanning with automated interconnect pipelining at the inter- and intra-FPGA levels to ensure high frequency. We have tested TAPA-CS on our multi-FPGA testbed where the FPGAs communicate through a high-speed 100Gbps Ethernet infrastructure. We have evaluated the performance and scalability of our tool on designs, including systolic-array based convolutional neural networks (CNNs), graph processing workloads such as page rank, stencil applications like the Dilate kernel, and K-nearest neighbors (KNN). TAPA-CS has the potential to accelerate development of increasingly complex and large designs on the low power and reconfigurable FPGAs.
Paper Structure (23 sections, 5 equations, 17 figures, 10 tables)

This paper contains 23 sections, 5 equations, 17 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Motivating Example of TAPA-CS
  4. TAPA-CS Design
  5. Problem Formulation
  6. Key Steps
  7. Inter-FPGA Module Mapping
  8. Inter-FPGA Communication
  9. Intra-FPGA Module Mapping
  10. Interconnect Pipelining
  11. Evaluation
  12. Benchmarks and Baselines
  13. Stencil
  14. Page Rank
  15. KNN
  16. ...and 8 more sections

Figures (17)

  • Figure 1: Network-Connected FPGAs
  • Figure 2: Architecture examples of modern FPGAs.
  • Figure 3: (A) Typical FPGA compilation flow, (B) Additions by TAPA-CS are highlighted in blue.
  • Figure 4: (A) Topology of the KNN application as found by the graph extraction step of TAPA-CS. Here, circles are compute modules (explained in Section \ref{['motivating-example']}), and hexagons indicate HBM access. (B) Partition found by TAPA-CS is indicated by the dashed line.
  • Figure 5: Key Steps in TAPA-CS
  • ...and 12 more figures