Jenga: Responsive Tiered Memory Management without Thrashing

TL;DR

Jenga tackles the dual challenges of banding hot data into a fast memory tier and avoiding thrashing by introducing a context-based allocator and smooth hotness tracking. The allocator groups objects by calling context and size to improve co-location of hot data within pages, while the hotness tracker decays gradually, decoupled from heavy-weight cooling to provide timely, stable migrations. Empirical evaluation across ten diverse applications shows Jenga achieving up to ~28% faster performance than the best prior tiered memory system when fast-tier capacity matches the working set, with CPU overhead <3% and memory overhead <0.3%; strong memory savings are observed as well. The work demonstrates practical gains for multi-tenant datacenters and discusses expected behavior on real CXL hardware, highlighting Jenga’s robust hit-rate and migration balance as a key practical advantage for heterogeneous memory environments.

Abstract

A heterogeneous memory has a single address space with fast access to some addresses (a fast tier of DRAM) and slow access to other addresses (a capacity tier of CXL-attached memory or NVM). A tiered memory system aims to maximize the number of accesses to the fast tier via page migrations between the fast and capacity tiers. Unfortunately, previous tiered memory systems can perform poorly due to (1) allocating hot and cold objects in the same page and (2) abrupt changes in hotness measurements that lead to thrashing. This paper presents Jenga, a tiered memory system that addresses both problems. Jenga's memory allocator uses a novel context-based page allocation strategy. Jenga's accurate measurements of page hotness enable it to react to memory access behavior changes in a timely manner while avoiding thrashing. Compared to the best previous tiered memory system, Jenga runs memory-intensive applications 28% faster across 10 applications, when the fast tier capacity matches the working set size, at a CPU overhead of <3% of a single core and a memory overhead of <0.3%

Figures (12)

• Figure 1: Application heatmaps. Darker colors indicate more accesses to a given memory range, during a given time period. GAPBS-BC, GAPBS-PR, FlexKVS, Graph500 and Liblinear create large objects and exhibit working set changes. XSBench, 603.bwaves_s and 654.roms_s contain a stable working set. Btree and Silo primarily allocate small objects.
• Figure 2: Impact of object grouping. This figure shows the CDF of memory accesses to pages for a read-only Btree workload, based on different object grouping strategies. A hypothetical popularity-based allocator packs all the popular objects together spatially, achieves a higher number of memory accesses in a fewer number of pages, followed by context-based grouping (§\ref{['sec:design:allocation']}).
• Figure 3: FlexKVS instantaneous throughput. Instantaneous performance for FlexKVS across tiered memory systems. There is a phase change at 50 seconds. No previous system achieves both high performance in a stable phase and timely adaptation to phase changes.
• Figure 4: Overview. Jenga uses a hybrid allocator for grouping objects (\ref{['sec:design:allocation']}) and smooth hotness tracking for adapting to workload changes while controlling migrations (\ref{['sec:design:migration']}).
• Figure 5: Calling context affinity. This figure shows the access pattern of different objects of the same size, but allocated from different calling contexts for the Btree application. Objects belonging to calling context 1 have a different popularity compared to objects belonging to calling contexts 2 & 3.