Survey of Disaggregated Memory: Cross-layer Technique Insights for Next-Generation Datacenters

TL;DR

This survey addresses the memory bottlenecks in modern data-centric workloads by examining disaggregated memory (DM) as a scalable, elastic memory pool that decouples CPU and memory resources. It presents a comprehensive cross-layer taxonomy, spanning hardware devices, DM architectures, OS/runtime adaptations, and application-level optimizations, to illuminate design tradeoffs and performance implications. The authors synthesize a wide range of techniques across architecture, system software, and applications, providing a sequence of design principles and future directions, including CXL-based memory pools, heterogeneous memory integration, and scalable memory orchestration. The work aims to guide datacenter designers toward elastic, efficient, and high-performance memory systems capable of supporting next-generation AI workloads and large-scale models.

Abstract

The growing scale of data requires efficient memory subsystems with large memory capacity and high memory performance. Disaggregated architecture has become a promising solution for today's cloud and edge computing for its scalability and elasticity. As a critical part of disaggregation, disaggregated memory faces many design challenges in many dimensions, including hardware scalability, architecture structure, software system design, application programmability, resource allocation, power management, etc. These challenges inspire a number of novel solutions at different system levels to improve system efficiency. In this paper, we provide a comprehensive review of disaggregated memory, including the methodology and technologies of disaggregated memory system foundation, optimization, and management. We study the technical essentials of disaggregated memory systems and analyze them from the hardware, architecture, system, and application levels. Then, we compare the design details of typical cross-layer designs on disaggregated memory. Finally, we discuss the challenges and opportunities of future disaggregated memory works that serve better for next-generation elastic and efficient datacenters.

Figures (8)

• Figure 1: This survey focuses on disaggregated memory(DM), which acts as an additional layer in the existing memory hierarchy. The survey mainly studies the design goal, design level, and key design points of the DM system.
• Figure 2: Overview of this survey.
• Figure 3: Memory bandwidth of different far memory devices, including memory, storage and network card sumsung-ssdnvidia-dpunvidia-rdma.
• Figure 4: Memory capacity of different far memory devices, including memory, storage and network card.
• Figure 5: Power usage of different far memory devices, including memory, storage and network card.