TCDM Burst Access: Breaking the Bandwidth Barrier in Shared-L1 RVV Clusters Beyond 1000 FPUs

TL;DR

The paper tackles bandwidth bottlenecks in large shared-L1 memory clusters with vector cores by introducing TCDM Burst Access, a software-transparent mechanism that uses 32-bit burst requests and widened response data channels to mitigate interconnect contention in hierarchical crossbars. The architecture consists of a Burst Sender that aggregates parallel requests and a Burst Manager that adapts burst traffic to banks, enabling parallel responses to be merged into wider transfers configurable via GF2/GF4. Validated on MemPool-Spatz testbeds with 16, 256, and 1024 FPUs, Burst Access achieves up to $118\%$–$226\%$ improvements in hierarchical bandwidth and substantial kernel performance gains (up to $2.76\times$) with modest area overhead (<$8\%$) and energy efficiency benefits up to $1.9\times$ for memory-bound workloads. The solution scales across multi-level interconnects and maintains feasibility in real silicon (12-nm FinFET), with open-source availability for broader adoption. Overall, Burst Access enables ultra-high bandwidth utilization in vector-dense L1-memory clusters, supporting efficient scaling beyond 1000 FPUs.

Abstract

As computing demand and memory footprint of deep learning applications accelerate, clusters of cores sharing local (L1) multi-banked memory are widely used as key building blocks in large-scale architectures. When the cluster's core count increases, a flat all-to-all interconnect between cores and L1 memory banks becomes a physical implementation bottleneck, and hierarchical network topologies are required. However, hierarchical, multi-level intra-cluster networks are subject to internal contention which may lead to significant performance degradation, especially for SIMD or vector cores, as their memory access is bursty. We present the TCDM Burst Access architecture, a software-transparent burst transaction support to improve bandwidth utilization in clusters with many vector cores tightly coupled to a multi-banked L1 data memory. In our solution, a Burst Manager dispatches burst requests to L1 memory banks, multiple 32b words from burst responses are retired in parallel on channels with parametric data-width. We validate our design on a RISC-V Vector (RVV) many-core cluster, evaluating the benefits on different core counts. With minimal logic area overhead (less than 8%), we improve the bandwidth of a 16-, a 256-, and a 1024--Floating Point Unit (FPU) baseline clusters, without Tightly Coupled Data Memory (TCDM) Burst Access, by 118%, 226%, and 77% respectively. Reaching up to 80% of the cores-memory peak bandwidth, our design demonstrates ultra-high bandwidth utilization and enables efficient performance scaling. Implemented in 12-nm FinFET technology node, compared to the serialized access baseline, our solution achieves up to 1.9x energy efficiency and 2.76x performance in real-world kernel benchmarkings.

Figures (5)

• Figure 1: Conflicts to shared interconnection resources reduce the interconnection bandwidth in vector many-core shared memory processors. The number on the request indicates its target bank.
• Figure 2: MP64Spatz4's Tile level architectural schematic with burst and GF4. The increased data-width response channels are marked in red.
• Figure 3: The roofline plots on original and burst-enabled configurations on MP4Spatz4 (left), MP64Spatz4 (middle), and MP128Spatz8 (right). The hierarchical average bandwidth is shown in dashed lines in the graph, the ideal no-contention bandwidth and maximum achievable performance are in solid black lines.
• Figure 4: Placed-and-routed layout annotated Group- and Tile-level view of GF4 design on MP64Spatz4 cluster
• Figure 5: Area (left) and power (right) breakdown for the MemPool64Spatz4 clusters. Area and power extracted in 12-nm technology, at $TT@910MHz$, executing kernel.