Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CXL and the Return of Scale-Up Database Engines

Alberto Lerner, Gustavo Alonso

TL;DR

It is argued that CXL can have a broader impact beyond memory expansion and deeply affect the architecture of data-intensive systems, and the associated research challenges.

Abstract

The trend toward specialized processing devices such as TPUs, DPUs, GPUs, and FPGAs has exposed the weaknesses of PCIe in interconnecting these devices and their hosts. Several attempts have been proposed to improve, augment, or downright replace PCIe, and more recently, these efforts have converged into a standard called Compute Express Link (CXL). CXL is already on version 2.0 in terms of commercial availability, but its potential to radically change the conventional server architecture has only just started to surface. For example, CXL can increase the bandwidth and quantity of memory available to any single machine beyond what that machine can originally provide, most importantly, in a manner that is fully transparent to software applications. We argue, however, that CXL can have a broader impact beyond memory expansion and deeply affect the architecture of data-intensive systems. In a nutshell, while the cloud favored scale-out approaches that grew in capacity by adding full servers to a rack, CXL brings back scale-up architectures that can grow by fine-tuning individual resources, all while transforming the rack into a large shared-memory machine. In this paper, we describe why such architectural transformations are now possible, how they benefit emerging heterogeneous hardware platforms for data-intensive systems, and the associated research challenges.

CXL and the Return of Scale-Up Database Engines

TL;DR

It is argued that CXL can have a broader impact beyond memory expansion and deeply affect the architecture of data-intensive systems, and the associated research challenges.

Abstract

The trend toward specialized processing devices such as TPUs, DPUs, GPUs, and FPGAs has exposed the weaknesses of PCIe in interconnecting these devices and their hosts. Several attempts have been proposed to improve, augment, or downright replace PCIe, and more recently, these efforts have converged into a standard called Compute Express Link (CXL). CXL is already on version 2.0 in terms of commercial availability, but its potential to radically change the conventional server architecture has only just started to surface. For example, CXL can increase the bandwidth and quantity of memory available to any single machine beyond what that machine can originally provide, most importantly, in a manner that is fully transparent to software applications. We argue, however, that CXL can have a broader impact beyond memory expansion and deeply affect the architecture of data-intensive systems. In a nutshell, while the cloud favored scale-out approaches that grew in capacity by adding full servers to a rack, CXL brings back scale-up architectures that can grow by fine-tuning individual resources, all while transforming the rack into a large shared-memory machine. In this paper, we describe why such architectural transformations are now possible, how they benefit emerging heterogeneous hardware platforms for data-intensive systems, and the associated research challenges.
Paper Structure (16 sections, 3 figures)

This paper contains 16 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Multicores and Memory Coherency
  4. Coherent and Non-Coherent Data Transfers
  5. CXL Status and Availability
  6. CXL Performance Characterization
  7. CXL as Networking
  8. CXL Scalability and Fault Tolerance
  9. Shared-Memory Architectures
  10. Memory Expansion
  11. Memory Pooling
  12. Memory Sharing
  13. Near-Data Processing
  14. Heterogeneous Architectures
  15. Related Efforts
  16. ...and 1 more sections

Figures (3)

  • Figure 1: (a) Peripherals connected using PCIe cards are outside the coherency domain, even if they contain memory. (b) CXL is a set of protocols that allow components on the peripheral to join the CPU coherency domain.
  • Figure 2: CXL allows different memory expansion architectures: (a) local expansion and (b) far-memory pooling, and (c) full-rack disaggregation. The latter two are achieved through specialized CXL switches.
  • Figure 3: Near-data processing under CXL: (a) A processor or FPGA managing the expanded memory can be co-opted to execute a portion of a query. (b) A unique opportunity for acceleration exists through active memory regions (AMR).