AgileWatts: An Energy-Efficient CPU Core Idle-State Architecture for Latency-Sensitive Server Applications

TL;DR

The paper addresses the energy inefficiency of latency-sensitive datacenter workloads by redesigning the CPU core idle-state architecture. It introduces C6A and C6AE deep idle states and a three-pronged design—Units' Fast Power-Gating (UFPG), Cache Coherence and Sleep Mode (CCSM), and a continuously-on all-digital PLL (ADPLL)—to achieve nanosecond-scale wake-ups while preserving most deep-idle power savings. The work provides a detailed PPA analysis, implementation considerations, and validation across Memcached, Kafka, and MySQL, showing up to 71% core energy reduction with minimal performance impact and significant data-center cost savings. The findings suggest that deep idle states can be effectively leveraged in latency-critical servers, enabling energy proportionality at low utilization and motivating adoption in modern server CPUs.

Abstract

User-facing applications running in modern datacenters exhibit irregular request patterns and are implemented using a multitude of services with tight latency requirements. These characteristics render ineffective existing energy conserving techniques when processors are idle due to the long transition time from a deep idle power state (C-state). While prior works propose management techniques to mitigate this inefficiency, we tackle it at its root with AgileWatts (AW): a new deep C-state architecture optimized for datacenter server processors targeting latency-sensitive applications. AW is based on three key ideas. First, AW eliminates the latency overhead of saving/restoring the core context (i.e., micro-architectural state) when powering-off/-on the core in a deep idle power state by i) implementing medium-grained power-gates, carefully distributed across the CPU core, and ii) retaining context in the power-ungated domain. Second, AW eliminates the flush latency overhead (several tens of microseconds) of the L1/L2 caches when entering a deep idle power state by keeping L1/L2 cache content power-ungated. A minimal control logic also remains power-ungated to serve cache coherence traffic (i.e., snoops) seamlessly. AW implements sleep-mode in caches to reduce caches leakage power consumption and lowers a core voltage to the minimum operational voltage level to minimize the leakage power of the power-ungated domain. Third, using a state-of-the-art power efficient all-digital phase-locked loop (ADPLL) clock generator, AW keeps the PLL active and locked during the idle state, further cutting precious microseconds of wake-up latency at a negligible power cost. Our evaluation with an accurate simulator calibrated against an Intel Skylake server shows that AW reduces the energy consumption of Memcached by up to 71% (35% on average) with up to 1% performance degradation.

Figures (13)

• Figure 1: An Intel Skylake server core slice skx_die_annotations. The core is bordered with green, and the AVX-512 and L2 extensions kumar2017intel (unavailable in client CPU cores) are bordered in pink. The 256-bit and 512-bit AVX units have separate power gates mandelblat2015technologyintel_avx512fayneh20164haj2021ichannels, as shown in the figure.
• Figure 2: Staggered power-gate wake-up by daisy-chaining the control signals of the power-gating switches.
• Figure 3: Entry and Exit Flows for (a) C1 and (b) C6.
• Figure 4: Medium-grain PG for the majority (area shaded in red) of the core units, excluding the L1 and L2 caches and their controllers.
• Figure 5: Context retention techniques AW uses when power-gating a unit: (a) Placing context in the core ungated power domain; (b) placing SRAM with context (e.g., microcode patch) in the core ungated power-domain; (c) using SRPG cells for distributed context.