OBASE: Object-Based Address-Space Engineering to Improve Memory Tiering

TL;DR

OBASE, a compiler-runtime system for unmanaged languages that serves as an object-aware frontend for page-aware OS backends and migrates objects at runtime using a lock-free protocol that is safe under concurrency is presented.

Abstract

Hardware and OS mechanisms for memory tiering are widely deployed, yet datacenters still overprovision DRAM. The root cause is hotness fragmentation: allocators place objects by size rather than access pattern, so hot and cold objects become interleaved within the same pages. A single hot object marks its page as active, trapping surrounding cold data in expensive DRAM. Our analysis of Google production workloads shows that up to 97% of the bytes in active pages are cold and unreclaimable. We propose address-space engineering: dynamically reorganizing virtual memory so that hot objects cluster into uniformly hot pages and cold objects into uniformly cold pages. We present OBASE, a compiler-runtime system for unmanaged languages that serves as an object-aware frontend for page-aware OS backends. OBASE tracks accesses via lightweight pointer instrumentation and migrates objects at runtime using a lock-free protocol that is safe under concurrency. By reorganizing the address space, OBASE enables unmodified backends (kswapd, TMO, TPP, Memtis) to tier memory effectively. Across ten concurrent data structures, six backends, and production traces from Meta and Twitter, OBASE improves page utilization by 2-4x and reduces memory footprint by up to 70%, with only 2-5% overhead.

Figures (12)

• Figure 1: The granularity gap in memory access of Google workloads.Dark Grey: The percentage of total memory bytes actually accessed. Light Grey: The percentage of total memory pages accessed. Red: The page utilization (calculated as Bytes Accessed / Pages Accessed, see §\ref{['sec:quantify-fragmentation']} for more details).
• Figure 2: Low Per-Page Utilization in Google Production Workloads.CDFs of page utilization for six widely deployed applications, shown for all pages combined (left), separately for 4KB pages (center), and 2MB pages (right).
• Figure 3: Temporal Evolution of Key Hotness.Heatmaps showing access frequency (log scale) for the top 5M keys over logical time buckets (10M operations each). If hotness were static, we would observe continuous vertical bands on the left side of each plot. Instead, we see shifting phases (Meta) and intermittent bursts (Twitter), demonstrating that the hot working set evolves continuously.
• Figure 4: OBASE Overview.OBASE acts as a frontend that organizes the virtual address space into hot and cold regions. The Object Collector monitors access via Guides and SODA, migrates objects using SAMA, and presents a re-organized address space to the OS backend.
• Figure 5: Object Migration State Diagram.Objects transition between heaps based on access intensity. The Object Collector promotes accessed objects to HOT and demotes inactive objects to COLD, allowing SAMA to apply differential madvise policies to each region if required.