Cohet: A CXL-Driven Coherent Heterogeneous Computing Framework with Hardware-Calibrated Full-System Simulation

TL;DR

Cohet introduces a CXL-driven coherent heterogeneous computing framework and SimCXL, a full-system simulator calibrated against real hardware to enable software–hardware co-design. By unifying CPU and XPU memories into a coherent pool and exposing standard interfaces, Cohet simplifies programming while delivering substantial latency and bandwidth benefits via CXL.cache over PCIe-based approaches. The work demonstrates two killer apps, RAO and RPC, showing significant performance improvements and validating the framework through detailed latency/bandwidth calibrations and application-level evaluations. Collectively, the study provides a practical open platform for exploring coherent heterogeneous computing and lays groundwork for future CXL-enabled systems.

Abstract

Conventional heterogeneous computing systems built on PCIe interconnects suffer from inefficient fine-grained host-device interactions and complex programming models. In recent years, many proprietary and open cache-coherent interconnect standards have emerged, among which compute express link (CXL) prevails in the open-standard domain after acquiring several competing solutions. Although CXL-based coherent heterogeneous computing holds the potential to fundamentally transform the collaborative computing mode of CPUs and XPUs, research in this direction remains hampered by the scarcity of available CXL-supported platforms, immature software/hardware ecosystems, and unclear application prospects. This paper presents Cohet, the first CXL-driven coherent heterogeneous computing framework. Cohet decouples the compute and memory resources to form unbiased CPU and XPU pools which share a single unified and coherent memory pool. It exposes a standard malloc/mmap interface to both CPU and XPU compute threads, leaving the OS dealing with smart memory allocation and management of heterogeneous resources. To facilitate Cohet research, we also present a full-system cycle-level simulator named SimCXL, which is capable of modeling all CXL sub-protocols and device types. SimCXL has been rigorously calibrated against a real CXL testbed with various CXL memory and accelerators, showing an average simulation error of 3%. Our evaluation reveals that CXL.cache reduces latency by 68% and increases bandwidth by 14.4x compared to DMA transfers at cacheline granularity. Building upon these insights, we demonstrate the benefits of Cohet with two killer apps, which are remote atomic operation (RAO) and remote procedure call (RPC). Compared to PCIe-NIC design, CXL-NIC achieves a 5.5 to 40.2x speedup for RAO offloading and an average speedup of 1.86x for RPC (de)serialization offloading.

Figures (18)

• Figure 1: PCIe-based heterogeneous computing architecture with CPU-biased computing and discrete physical memories.
• Figure 2: Emerging CXL-based coherent heterogeneous computing architecture with unbiased cooperative computing and a coherent unified memory pool.
• Figure 3: Vertical stack of the Cohet computing framework.
• Figure 4: Comparison between CUDA and Cohet programming models using AXPY operation as an example.
• Figure 5: Architecture of the proposed SimCXL simulator.