Coach: Exploiting Temporal Patterns for All-Resource Oversubscription in Cloud Platforms

TL;DR

Coach addresses persistent cloud underutilization by exploiting predictable temporal patterns in VM workloads to oversubscribe all resources. It introduces CoachVM, a general-purpose VM type with guaranteed and oversubscribed portions and a time-window based predictive scheduler that prioritizes memory performance through PA/VA backing and zNUMA mapping. Evaluations on Azure-scale traces and a production server show up to $\sim 26\%$ more VMs can be hosted with minimal performance degradation and manageable overhead, aided by contention monitoring and proactive mitigations. The work demonstrates a practical path to higher consolidation without workload changes.

Abstract

Cloud platforms remain underutilized despite multiple proposals to improve their utilization (e.g., disaggregation, harvesting, and oversubscription). Our characterization of the resource utilization of virtual machines (VMs) in Azure reveals that, while CPU is the main underutilized resource, we need to provide a solution to manage all resources holistically. We also observe that many VMs exhibit complementary temporal patterns, which can be leveraged to improve the oversubscription of underutilized resources. Based on these insights, we propose Coach: a system that exploits temporal patterns for all-resource oversubscription in cloud platforms. Coach uses long-term predictions and an efficient VM scheduling policy to exploit temporally complementary patterns. We introduce a new general-purpose VM type, called CoachVM, where we partition each resource allocation into a guaranteed and an oversubscribed portion. Coach monitors the oversubscribed resources to detect contention and mitigate any potential performance degradation. We focus on memory management, which is particularly challenging due to memory's sensitivity to contention and the overhead required to reassign it between CoachVMs. Our experiments show that Coach enables platforms to host up to ~26% more VMs with minimal performance degradation.

Figures (21)

• Figure 1: Examples of the causes of low resource utilization.
• Figure 2: Percentage of resource hours consumed by VMs lasting longer than VM duration (left) and the percentage of VMs lasting more than VM duration (right).
• Figure 3: Resource hours and number of VMs consumed by VMs larger than a size (cores and memory).
• Figure 4: Average stranding for different resource types with varying levels of hypothetical oversubscription.
• Figure 5: Percentage of time each resource is the bottleneck.