CoroAMU: Unleashing Memory-Driven Coroutines through Latency-Aware Decoupled Operations

TL;DR

The paper addresses high memory latency in data-center workloads using disaggregated memory by introducing CoroAMU, a hardware-software co-design that uses memory-centric coroutines. It combines an LLVM-based coroutine compiler with an enhanced AMU that supports decoupled memory operations and dynamic scheduling, augmented by a novel memory-guided branch predictor. Key contributions include novel compiler passes for context and request optimization, aggregation of memory operations, and bafin-based branch prediction, alongside architectural extensions enabling aggregated memory requests and await/asignal synchronization. Empirical results show average speedups of $1.51\times$ over state-of-the-art on Intel, and up to $3.39\times$ and $4.87\times$ improvements at memory latencies of $200\mathrm{ns}$ and $800\mathrm{ns}$ on FPGA-emulated disaggregated systems, underscoring the practical impact of hardware-software co-design for memory-latency hiding in modern data-center workloads.

Abstract

Modern data-intensive applications face memory latency challenges exacerbated by disaggregated memory systems. Recent work shows that coroutines are promising in effectively interleaving tasks and hiding memory latency, but they struggle to balance latency-hiding efficiency with runtime overhead. We present CoroAMU, a hardware-software co-designed system for memory-centric coroutines. It introduces compiler procedures that optimize coroutine code generation, minimize context, and coalesce requests, paired with a simple interface. With hardware support of decoupled memory operations, we enhance the Asynchronous Memory Unit to further exploit dynamic coroutine schedulers by coroutine-specific memory operations and a novel memory-guided branch prediction mechanism. It is implemented with LLVM and open-source XiangShan RISC-V processor over the FPGA platform. Experiments demonstrate that the CoroAMU compiler achieves a 1.51x speedup over state-of-the-art coroutine methods on Intel server processors. When combined with optimized hardware of decoupled memory access, it delivers 3.39x and 4.87x average performance improvements over the baseline processor on FPGA-emulated disaggregated systems under 200ns and 800ns latency respectively.

Figures (16)

• Figure 1: Coroutine usage based on software prefetching and its generated IR.
• Figure 2: Performance comparisons of serial and coroutine-based execution, normalized to serial. The red line shows the baseline performance with cross-NUMA memory accesses, and the green one shows the performance with a perfect cache.
• Figure 3: Performance breakdown of coroutine-optimized applications. The local memory part includes context-switching overhead.
• Figure 4: Basic concept of AMU.
• Figure 5: CoroAMU overview, with a compiler automatically generating coroutines and enhanced AMU hardware.