Workload Intelligence: Punching Holes Through the Cloud Abstraction

TL;DR

WI addresses the opacity of cloud interfaces by enabling bidirectional hints between workloads and the platform. It achieves this through a general, extensible hint framework and a scalable architecture of local and global managers that coordinate multiple optimizations. The authors characterize 188 real workloads, map ten optimizations to workload characteristics, and implement WI across Hadoop, Kubernetes, and OpenWhisk, showing substantial cost savings (average 48.8%) and carbon reductions, while preserving QoS. The results demonstrate that piercing the cloud abstraction with WI can simplify offerings, reduce costs, and enable dynamic, workload-aware optimization in practice.

Abstract

Today, cloud workloads are essentially opaque to the cloud platform. Typically, the only information the platform receives is the virtual machine (VM) type and possibly a decoration to the type (e.g., the VM is evictable). Similarly, workloads receive little to no information from the platform; generally, workloads might receive telemetry from their VMs or exceptional signals (e.g., shortly before a VM is evicted). The narrow interface between workloads and platforms has several drawbacks: (1) a surge in VM types and decorations in public cloud platforms complicates customer selection; (2) essential workload characteristics (e.g., low availability requirements, high latency tolerance) are often unspecified, hindering platform customization for optimized resource usage and cost savings; and (3) workloads may be unaware of potential optimizations or lack sufficient time to react to platform events. In this paper, we propose a framework, called Workload Intelligence (WI), for dynamic bi-directional communication between cloud workloads and cloud platform. Via WI, workloads can programmatically adjust their key characteristics, requirements, and even dynamically adapt behaviors like VM priorities. In the other direction, WI allows the platform to programmatically inform workloads about upcoming events, opportunities for optimization, among other scenarios. Because of WI, the cloud platform can drastically simplify its offerings, reduce its costs without fear of violating any workload requirements, and reduce prices to its customers on average by 48.8%.

Figures (5)

• Figure 1: Workload Intelligence overview.
• Figure 2: Workload Intelligence architecture showing three example cloud optimizations.
• Figure 3: Conflict resolution for competing resources.
• Figure 4: Slowdown and cost running the Big Data workload. The baseline (i.e., 1$\times$, 100%) is the median for Regular VMs.
• Figure 5: Cost savings breakdown by optimizations.