Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Opportunistic Scheduling for Optimal Spot Instance Savings in the Cloud

Neelkamal Bhuyan, Randeep Bhatia, Murali Kodialam, TV Lakshman

TL;DR

The paper develops a formal framework for scheduling delay-sensitive jobs across cheaper spot and expensive on-demand cloud instances, modeled as a $G/G/1$-style system with decision variables for joining queueing and maximal waiting times. It proves that under tight delay the optimal policy caps the queue length at one and derives the corresponding cost $\mathbb{E}[C]=k-(k-1)\mu\delta$, while under relaxed delay a knapsack-like structure governs a near-optimal greedy policy and a learnable fractional admission parameter $r^*$. The authors derive LP-based characterizations for the optimal waiting distributions, provide closed-form results for common inter-arrival distributions (finite-support and exponential spot arrivals), and propose an adaptive admission control algorithm that learns $r^*$ online. Experiments on preemptible GCP instances and $M/M/1/N$ configurations demonstrate convergence to near-optimal costs and adherence to delay constraints, confirming practical applicability in dynamic cloud environments.

Abstract

We study the problem of scheduling delay-sensitive jobs over spot and on-demand cloud instances to minimize average cost while meeting an average delay constraint. Jobs arrive as a general stochastic process, and incur different costs based on the instance type. This work provides the first analytical treatment of this problem using tools from queuing theory, stochastic processes, and optimization. We derive cost expressions for general policies, prove queue length one is optimal for low target delays, and characterize the optimal wait-time distribution. For high target delays, we identify a knapsack structure and design a scheduling policy that exploits it. An adaptive algorithm is proposed to fully utilize the allowed delay, and empirical results confirm its near-optimality.

Opportunistic Scheduling for Optimal Spot Instance Savings in the Cloud

TL;DR

The paper develops a formal framework for scheduling delay-sensitive jobs across cheaper spot and expensive on-demand cloud instances, modeled as a -style system with decision variables for joining queueing and maximal waiting times. It proves that under tight delay the optimal policy caps the queue length at one and derives the corresponding cost , while under relaxed delay a knapsack-like structure governs a near-optimal greedy policy and a learnable fractional admission parameter . The authors derive LP-based characterizations for the optimal waiting distributions, provide closed-form results for common inter-arrival distributions (finite-support and exponential spot arrivals), and propose an adaptive admission control algorithm that learns online. Experiments on preemptible GCP instances and configurations demonstrate convergence to near-optimal costs and adherence to delay constraints, confirming practical applicability in dynamic cloud environments.

Abstract

We study the problem of scheduling delay-sensitive jobs over spot and on-demand cloud instances to minimize average cost while meeting an average delay constraint. Jobs arrive as a general stochastic process, and incur different costs based on the instance type. This work provides the first analytical treatment of this problem using tools from queuing theory, stochastic processes, and optimization. We derive cost expressions for general policies, prove queue length one is optimal for low target delays, and characterize the optimal wait-time distribution. For high target delays, we identify a knapsack structure and design a scheduling policy that exploits it. An adaptive algorithm is proposed to fully utilize the allowed delay, and empirical results confirm its near-optimality.
Paper Structure (16 sections, 9 theorems, 55 equations, 5 figures, 1 table, 1 algorithm)

This paper contains 16 sections, 9 theorems, 55 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Model and Preliminaries
  3. Analytical model
  4. Cost per Job for a General System
  5. Optimal scheduling under low target delay $\delta$
  6. Optimal queue-length
  7. Optimal wait-time distribution
  8. Solving for common distributions
  9. Scheduling under high target delay $\delta$
  10. Near-optimal Heuristic Scheduling Policy
  11. Learning optimal scheduling on-the-fly
  12. Experiments
  13. Preemptible GCP instances
  14. $M/M/1/N$ system
  15. Related Work
  16. ...and 1 more sections

Key Result

Theorem 1

Consider a general job arrival process and a general spot arrival process $(G/G/1)$. The cost per job for any scheduling policy in steady state is where $\mathbbm{E}[A]: = \frac{1}{\lambda}$ is the average job inter-arrival time, $\mathbbm{E}[S_\mu]: = \frac{1}{\mu}$ is the average spot inter-arrival time and $\pi_0$ is the steady state probability of the queue being empty.

Figures (5)

  • Figure 1: Job Arrivals and Departures: Delay and Costs
  • Figure 2: Learning optimal scheduling $\delta < \frac{1}{\lambda + \mu}$
  • Figure 3: Learning optimal scheduling for $\lambda \delta > 1$
  • Figure 4: Learning optimal scheduling $\delta < \frac{1}{\lambda + \mu}$ for memoryless system
  • Figure 5: Learning optimal scheduling $\lambda\delta>1$ for memoryless system

Theorems & Definitions (20)

  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Corollary 1
  • proof
  • Corollary 2
  • proof
  • ...and 10 more