How long can you sleep? Idle Time System Inefficiencies and Opportunities
Georgia Antoniou, Haris Volos, Jawad Haj Yahya, Yiannakis Sazeides
TL;DR
The paper addresses wasted idle power in modern latency-sensitive servers by quantifying idle opportunity through a model-based framework. It applies queueing-theory models $M/M/1$, $cxM/M/1$, and $M/M/c$ to estimate idle-time distributions at core and package levels, and validates these predictions against measurements from a real 2-socket server with synthetic workloads. Key contributions include quantifying unrealized idle opportunities, identifying sources of inefficiency such as idle-governor inaccuracy and high transition overheads, and offering design guidance for hardware/software optimizations and early-stage exploration of configurations (e.g., core counts per package). The work enables more energy-efficient server design by pinpointing feasible deep-idle states and informing future power-management strategies, with extensibility to other system characteristics like interrupt overheads.
Abstract
This work introduces a model-based framework that reveals the idle opportunity of modern servers running latency-critical applications. Specifically, three queuing models, M/M/1, cxM/M/1, and M/M/c, are used to estimate the theoretical idle time distribution at the CPU core and system (package) level. A comparison of the actual idleness of a real server and that from the theoretical models reveals significant missed opportunities to enter deep idle states. This inefficiency is attributed to the idle-governor inaccuracy and the high latency to transition to/from legacy deep-idle states. The proposed methodology offers the means for an early-stage design exploration and insights into idle time behavior and opportunities for varying server system configurations and load.