Taming Offload Overheads in a Massively Parallel Open-Source RISC-V MPSoC: Analysis and Optimization

TL;DR

This paper analyzes offload overheads in a massively parallel open-source RISC-V MPSoC (Occamy) and shows how co-designing hardware (multicast-capable interconnect) with offload software (bare-metal, host-centric offloading) and a job-completion unit dramatically reduces overheads. Through cycle-accurate RTL simulations, it provides a detailed breakdown of each offload phase, derives analytic runtimes that match actual measurements within 15%, and demonstrates up to a 2.3× speedup gains with the multicast extension. The work reveals that offload overheads grow with the number of accelerator cores, and that multiprocessing of offload tasks is most beneficial for fine-grained workloads, restoring a large fraction of the ideal speedups for representative kernels. The contributions deliver actionable insights for designing scalable, energy-efficient heterogeneous systems and offer open benchmarks and models to guide future research and optimization.

Abstract

Heterogeneous multi-core architectures combine on a single chip a few large, general-purpose host cores, optimized for single-thread performance, with (many) clusters of small, specialized, energy-efficient accelerator cores for data-parallel processing. Offloading a computation to the many-core acceleration fabric implies synchronization and communication overheads which can hamper overall performance and efficiency, particularly for small and fine-grained parallel tasks. In this work, we present a detailed, cycle-accurate quantitative analysis of the offload overheads on Occamy, an open-source massively parallel RISC-V based heterogeneous MPSoC. We study how the overheads scale with the number of accelerator cores. We explore an approach to drastically reduce these overheads by co-designing the hardware and the offload routines. Notably, we demonstrate that by incorporating multicast capabilities into the Network-on-Chip of a large (200+ cores) accelerator fabric we can improve offloaded application runtimes by as much as 2.3x, restoring more than 70% of the ideally attainable speedups. Finally, we propose a quantitative model to estimate the runtime of selected applications accounting for the offload overheads, with an error consistently below 15%.

Figures (12)

• Figure 1: Arm Generic Interrupt Controller (GIC) Architecture
• Figure 2: Simplified block diagram of the Occamy .
• Figure 3: Example trace of the job phases in a 2-cluster Occamy system. Segments are drawn to qualitatively reflect actual observations. Dashed arrows represent interrupt latencies and red lines represent synchronization barriers. Brackets at the bottom and top of the figure group the phases which belong to the same fundamental offload task described in section \ref{['sec:offload-tasks']}.
• Figure 4: Block diagram of a 4x4 AXI XBAR.
• Figure 5: Encoding of multiple addresses.