Tuning Fast Memory Size based on Modeling of Page Migration for Tiered Memory

TL;DR

Tuna addresses the challenge of selecting fast memory size in tiered memory by modeling the impact of page migration on application performance. It combines offline micro-benchmarking with a Faiss-indexed performance database to estimate the minimal fast-memory size that satisfies a user-defined performance loss target, and implements a runtime memory-tuning controller (MTC) that adjusts fast memory via Linux watermark values. The approach demonstrates average fast-memory savings of $8.5\%$ (up to $16\%$) across representative workloads while staying within a $\tau=5\%$ performance loss, outperforming prior Pond-based results for the same workloads. The key contributions include the Tuna system architecture, the micro-benchmark design, the performance-database methodology, and the runtime integration that adapts fast-memory size to workload characteristics with minimal overhead.

Abstract

Tiered memory, built upon a combination of fast memory and slow memory, provides a cost-effective solution to meet ever-increasing requirements from emerging applications for large memory capacity. Reducing the size of fast memory is valuable to improve memory utilization in production and reduce production costs because fast memory tends to be expensive. However, deciding the fast memory size is challenging because there is a complex interplay between application characterization and the overhead of page migration used to mitigate the impact of limited fast memory capacity. In this paper, we introduce a system, Tuna, to decide fast memory size based on modeling of page migration. Tuna uses micro-benchmarking to model the impact of page migration on application performance using three metrics. Tuna decides the fast memory size based on offline modeling results and limited information on workload telemetry. Evaluating with common big-memory applications and using 5% as the performance loss target, we show that Tuna in combination with a page management system (TPP) saves fast memory by 8.5% on average (up to 16%). This is in contrast to the 5% saving in fast memory reported by Microsoft Pond for the same workloads (BFS and SSSP) and the same performance loss target.

Figures (8)

• Figure 1: BFS performance with various fast memory sizes and a page management system (TPP) in place. Percentage numbers in the $x$ axis are the percentage of the original fast memory size. "FM" stands for fast memory.
• Figure 2: Overview of Tuna
• Figure 3: Fast memory variance: XSBench
• Figure 4: Fast memory variance: BFS
• Figure 5: Fast memory variance: PageRank