Tidying Up the Address Space

TL;DR

The paper addresses memory tiering inefficiencies caused by hotness fragmentation, where hot and cold objects share pages and hinder reclaimability. It introduces address-space engineering and the HADES frontend to dynamically reorganize object placement by access intensity, decoupling it from page-level reclamation and enabling unmodified backends to reclaim memory more aggressively. Through evaluation on ten pointer-based data structures, HADES demonstrates up to 70% memory reduction with only 3–5% performance overhead, validating a practical approach to improve memory tiering without OS or backend changes. The work highlights the resulting improvements in CPU efficiency and reduced server footprint, offering a viable path for more cost-effective and environmentally sustainable datacenters.

Abstract

Memory tiering in datacenters does not achieve its full potential due to hotness fragmentation -- the intermingling of hot and cold objects within memory pages. This fragmentation prevents page-based reclamation systems from distinguishing truly hot pages from pages containing mostly cold objects, fundamentally limiting memory efficiency despite highly skewed accesses. We introduce address-space engineering: dynamically reorganizing application virtual address spaces to create uniformly hot and cold regions that any page-level tiering backend can manage effectively. HADES demonstrates this frontend/backend approach through a compiler-runtime system that tracks and migrates objects based on access patterns, requiring minimal developer intervention. Evaluations across ten data structures achieve up to 70% memory reduction with 3% performance overhead, showing that address space engineering enables existing reclamation systems to reclaim memory aggressively without performance degradation.

Figures (10)

• Figure 1: Address-Space Engineering.Hotness fragmentation (top) intermixes hot (red) and cold (blue) objects, making pages difficult to reclaim. Address-space engineering (bottom) reorganizes objects into uniformly hot and cold regions, enabling efficient page-level management.
• Figure 2: Page Utilization.Redis, Memcached, and MongoDB Page Utilization for 360s epochs running a YCSB-C (read-only) workload with a Zipfian distribution. A small fraction of bytes per page are accessed, and more pages are touched than necessary.
• Figure 3: Unreclaimable Memory.YCSB-C with Zipfian distribution running on Redis shows memory used (RSS), pages needed (Touched Pages), and cachelines needed (Touched Cachelines). Only 0.5 MiB of actual data is required whereas 1.2 GiB remains resident. The gap represents theoretically reclaimable memory that current systems cannot efficiently recover.
• Figure 4: HADES Overview.As a front-end, HADES makes object placement decisions to organize the virtual address space into hot and cold regions. This enables any back-end to make more effective page placement decisions between DRAM and tiered storage.
• Figure 5: Object Migration State Diagram.An object starts in NEW heap and migrates to HOT or COLD based on access patterns and if its Consecutive Inactive Windows (CIW) is greater than a threshold ($C_t$), with heap properties adjusted through madvise flags.