Exploiting Spot Instances for Time-Critical Cloud Workloads Using Optimal Randomized Strategies

TL;DR

This work tackles deadline-aware online scheduling in hybrid clouds where jobs may run on cheap but revocable spot instances or reliable on-demand instances costing $K$ per unit time, with a hard deadline. It proves a fundamental $\Omega(K)$ lower bound for any deterministic policy and introduces Ross, a randomized online scheduler, achieving a provably optimal $\sqrt{K}$ competitive ratio under practical deadlines. The authors establish formal guarantees (Theorem 1–3) and validate Ross on real Azure, AWS, and Skypilot traces, demonstrating significant cost savings (up to $30\%$) while preserving deadline guarantees across varying spot conditions. The results show that injecting a controlled, randomized on-demand interval during slack balances cost-efficiency with reliability, enabling effective use of spot resources in time-critical cloud workloads.

Abstract

This paper addresses the challenge of deadline-aware online scheduling for jobs in hybrid cloud environments, where jobs may run on either cost-effective but unreliable spot instances or more expensive on-demand instances, under hard deadlines. We first establish a fundamental limit for existing (predominantly-) deterministic policies, proving a worst-case competitive ratio of $Ω(K)$, where $K$ is the cost ratio between on-demand and spot instances. We then present a novel randomized scheduling algorithm, ROSS, that achieves a provably optimal competitive ratio of $\sqrt{K}$ under reasonable deadlines, significantly improving upon existing approaches. Extensive evaluations on real-world trace data from Azure and AWS demonstrate that ROSS effectively balances cost optimization and deadline guarantees, consistently outperforming the state-of-the-art by up to $30\%$ in cost savings, across diverse spot market conditions.

Figures (5)

• Figure 1: Schematic of ross
• Figure 2: Spotlake Trace Azure D48as-v5-AU-east-1 First row: % savings w.r.t on-demand as deadline get stricter. Second row: % extra cost over opt as $K$ increases. Left plot presents loose deadlines and right plot presents strict deadlines.
• Figure 3: Spotlake Trace AWS-EC2 c3large-us.east-1 First row: % savings w.r.t on-demand as deadline get stricter. Second row: % extra cost over opt as $K$ increases. Left plot presents loose deadlines and right plot presents strict deadlines.
• Figure 4: Skypilot Availability Trace us-west-2b-v100-8 First row: % savings w.r.t on-demand as deadline get stricter. Second row: % extra cost over opt as $K$ increases. Left plot presents loose deadlines and right plot presents strict deadlines.
• Figure 5: Skypilot Preemption Trace us-east-1f-v100-1 First row: % savings w.r.t on-demand as deadline get stricter. Second row: % extra cost over opt as $K$ increases. Left plot presents loose deadlines and right plot presents strict deadlines.

Theorems & Definitions (7)

• Definition 1
• Theorem 1
• Theorem 2
• Theorem 3
• proof : Proof of Theorem \ref{['thm:det_policies']}
• proof : Proof of Theorem 2
• proof : Proof of Theorem 3