A Full-System Simulation Framework for CXL-Based SSD Memory System

TL;DR

The paper tackles the memory capacity bottleneck in data-centric workloads by leveraging CXL-based SSDs within a memory pool. It presents CXL-SSD-Sim, a first open-source full-system simulator built on gem5 and SimpleSSD, augmented with a DRAM cache to reduce SSD latency and enable realistic system-level studies. Key contributions include a high-fidelity CXL.mem sub-protocol implementation, a kernel driver integration to bridge CXL devices with the OS, and a DRAM caching design with five replacement policies that mitigates granularity and latency mismatches. The framework is validated across five memory device configurations, demonstrating accurate latency and bandwidth behavior and enabling real-world workload analysis such as Viper benchmarks. This tool enables researchers to explore CXL memory pool architectures, OS impact, and storage-augmented memory systems with end-to-end simulation capabilities, addressing a critical gap in CXL research and development.

Abstract

Compute eXpress Link (CXL) is a promising technology for memory disaggregation and expansion. Especially, CXL makes it more effectively for large-capacity storage devices such as Solid State Drive (SSD) to be deployed in the memory pool. However, CXL-based SSDs are still in early stages, necessitating the development of reliable simulation tools. In this paper, we propose CXL-SSD-Sim, the first open-source full-system simulator designed to simulate CXL-based SSD memory system. Constructed on the foundation of gem5 and SimpleSSD, CXL-SSD-Sim extends an high fidelity SSD memory expander model along with the corresponding device driver. In addition, CXL-SSD-Sim models a DRAM layer as a caching mechanism for the SSD, meticulously engineered to counteract latency issues inherent to CXL-based SSD memory access. Experiments are performed among five different memory devices with CXL-SSD-Sim in aspect of latency, bandwidth and real-world benchmark performance. These experiments serve to underscore the efficacy of our simulation tool in providing a comprehensive analysis of CXL-based SSD memory systems. The CXL-SSD-Sim simulator is available at https://github.com/WangYaohuii/CXL-SSD-Sim.

Figures (6)

• Figure 1: Overall structure of the CXL-SSD-Sim extended memory device simulator.
• Figure 2: CXL-SSD device access.
• Figure 3: Bandwidth performance comparison across different memory devices.
• Figure 4: Latency performance comparison across different memory devices.
• Figure 5: QPS comparison for Viper 216B key-value pairs across different memory devices.