Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TeraPool: A Physical Design Aware, 1024 RISC-V Cores Shared-L1-Memory Scaled-up Cluster Design with High Bandwidth Main Memory Link

Yichao Zhang, Marco Bertuletti, Chi Zhang, Samuel Riedel, Diyou Shen, Bowen Wang, Alessandro Vanelli-Coralli, Luca Benini

TL;DR

TeraPool is a physically implementable, floating-point-capable RISC-V PEs scaled-up cluster design sharing a Multi-MegaByte PE-to-L1-memory interconnect that achieves near-gigahertz frequencies and 0.80 V/25  the cost of a FP32 FMA.

Abstract

Shared L1-memory clusters of streamlined instruction processors (processing elements - PEs) are commonly used as building blocks in modern, massively parallel computing architectures (e.g. GP-GPUs). Scaling out these architectures by increasing the number of clusters incurs computational and power overhead, caused by the requirement to split and merge large data structures in chunks and move chunks across memory hierarchies via the high-latency global interconnect. Scaling up the cluster reduces buffering, copy, and synchronization overheads. However, the complexity of a fully connected cores-to-L1-memory crossbar grows quadratically with PE-count, posing a major physical implementation challenge. We present TeraPool, a physically implementable, >1000 floating-point-capable RISC-V PEs scaled-up cluster design, sharing a Multi-MegaByte >4000-banked L1 memory via a low latency hierarchical interconnect (1-7/9/11 cycles, depending on target frequency). Implemented in 12nm FinFET technology, TeraPool achieves near-gigahertz frequencies (910MHz) typical, 0.80 V/25C. The energy-efficient hierarchical PE-to-L1-memory interconnect consumes only 9-13.5pJ for memory bank accesses, just 0.74-1.1x the cost of a FP32 FMA. A high-bandwidth main memory link is designed to manage data transfers in/out of the shared L1, sustaining transfers at the full bandwidth of an HBM2E main memory. At 910MHz, the cluster delivers up to 1.89 single precision TFLOP/s peak performance and up to 200GFLOP/s/W energy efficiency (at a high IPC/PE of 0.8 on average) in benchmark kernels, demonstrating the feasibility of scaling a shared-L1 cluster to a thousand PEs, four times the PE count of the largest clusters reported in literature.

TeraPool: A Physical Design Aware, 1024 RISC-V Cores Shared-L1-Memory Scaled-up Cluster Design with High Bandwidth Main Memory Link

TL;DR

TeraPool is a physically implementable, floating-point-capable RISC-V PEs scaled-up cluster design sharing a Multi-MegaByte PE-to-L1-memory interconnect that achieves near-gigahertz frequencies and 0.80 V/25  the cost of a FP32 FMA.

Abstract

Shared L1-memory clusters of streamlined instruction processors (processing elements - PEs) are commonly used as building blocks in modern, massively parallel computing architectures (e.g. GP-GPUs). Scaling out these architectures by increasing the number of clusters incurs computational and power overhead, caused by the requirement to split and merge large data structures in chunks and move chunks across memory hierarchies via the high-latency global interconnect. Scaling up the cluster reduces buffering, copy, and synchronization overheads. However, the complexity of a fully connected cores-to-L1-memory crossbar grows quadratically with PE-count, posing a major physical implementation challenge. We present TeraPool, a physically implementable, >1000 floating-point-capable RISC-V PEs scaled-up cluster design, sharing a Multi-MegaByte >4000-banked L1 memory via a low latency hierarchical interconnect (1-7/9/11 cycles, depending on target frequency). Implemented in 12nm FinFET technology, TeraPool achieves near-gigahertz frequencies (910MHz) typical, 0.80 V/25C. The energy-efficient hierarchical PE-to-L1-memory interconnect consumes only 9-13.5pJ for memory bank accesses, just 0.74-1.1x the cost of a FP32 FMA. A high-bandwidth main memory link is designed to manage data transfers in/out of the shared L1, sustaining transfers at the full bandwidth of an HBM2E main memory. At 910MHz, the cluster delivers up to 1.89 single precision TFLOP/s peak performance and up to 200GFLOP/s/W energy efficiency (at a high IPC/PE of 0.8 on average) in benchmark kernels, demonstrating the feasibility of scaling a shared-L1 cluster to a thousand PEs, four times the PE count of the largest clusters reported in literature.
Paper Structure (23 sections, 6 equations, 14 figures, 6 tables)

This paper contains 23 sections, 6 equations, 14 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Shared-memory cluster scale-up vs. scale-out
  3. Single Cluster Scale-up
  4. Scale-out to Multiple Clusters
  5. interconnect Design
  6. Hierarchical Interconnect AMAT Model
  7. TeraPool interconnect Analysis
  8. TeraPool Cluster Architecture
  9. Processing Elements
  10. Cluster Design
  11. High Bandwidth Memory Link
  12. Hierarchical AXI Interconnect
  13. Modular DMA engine
  14. Main Memory
  15. Memory Mapping Scheme
  16. ...and 8 more sections

Figures (14)

  • Figure 1: Illustration of scaling-up and scaling-out in cluster(s) design.
  • Figure 2: The hierarchical Tile (building block) design with logarithmic-staged crossbar interconnect.
  • Figure 3: Routing Congestion of Logarithmic-Crossbar-Based Interconnect at Different Complexities (GF12nm, 13M).
  • Figure 4: Block diagram of the Snitch highlighting the LSU, and the LSU's transaction table marking outstanding load and stores.
  • Figure 5: Block diagram of Tile: 8 core-complexes, 2 shared FP-DIVSQRT units; a local crossbar for 1-cycle access to tightly coupled data memory (TCDM), and 7 Remote Req/Resp ports connect to higher-level hierarchies.
  • ...and 9 more figures