Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Probabilistic Workflow Scheduling for IaaS Clouds

Gabriele Russo Russo, Romolo Marotta, Flavio Cordari, Francesco Quaglia, Valeria Cardellini, Pierangelo Di Sanzo

TL;DR

This work tackles probabilistic DAG scheduling for IaaS clouds under deadlines and budgets, where task durations are uncertain and can vary across VM types. It introduces EPOSS, a MOHEFT-based, quantile-driven algorithm that uses binary search on task-time quantiles and Monte Carlo evaluation to meet probabilistic constraints on makespan while minimizing expected cost; it also provides a parallel (P-EPOSS) and a multi-objective (M-EPOSS) extension. Empirical results show EPOSS achieves 10–100× faster solution discovery than state-of-the-art probabilistic schedulers, with comparable or better cost and feasibility across diverse workflows, VM pools, and distributions, while handling per-type resource quotas. The approach demonstrates practical applicability for efficient, uncertainty-aware workflow scheduling in IaaS environments and suggests future work on automated task-model profiling and distribution fitting.

Abstract

The flexibility and the variety of computing resources offered by the cloud make it particularly attractive for executing user workloads. However, IaaS cloud environments pose non-trivial challenges in the case of workflow scheduling under deadlines and monetary cost constraints. Indeed, given the typical uncertain performance behavior of cloud resources, scheduling algorithms that assume deterministic execution times may fail, thus requiring probabilistic approaches. However, existing probabilistic algorithms are computationally expensive, mainly due to the greater complexity of the workflow scheduling problem in its probabilistic form, and they hardily scale with the size of the problem instance. In this article, we propose EPOSS, a novel workflow scheduling algorithm for IaaS cloud environments based on a probabilistic formulation. Our solution blends together the low execution latency of state-of-the-art scheduling algorithms designed for the case of deterministic execution times and the capability to enforce probabilistic constraints.Designed with computational efficiency in mind, EPOSS achieves one to two orders lower execution times in comparison with existing probabilistic schedulers. Furthermore, it ensures good scaling with respect to workflow size and number of heterogeneous virtual machine types offered by the IaaS cloud environment. We evaluated the benefits of our algorithm via an experimental comparison over a variety of workloads and characteristics of IaaS cloud environments.

Efficient Probabilistic Workflow Scheduling for IaaS Clouds

TL;DR

This work tackles probabilistic DAG scheduling for IaaS clouds under deadlines and budgets, where task durations are uncertain and can vary across VM types. It introduces EPOSS, a MOHEFT-based, quantile-driven algorithm that uses binary search on task-time quantiles and Monte Carlo evaluation to meet probabilistic constraints on makespan while minimizing expected cost; it also provides a parallel (P-EPOSS) and a multi-objective (M-EPOSS) extension. Empirical results show EPOSS achieves 10–100× faster solution discovery than state-of-the-art probabilistic schedulers, with comparable or better cost and feasibility across diverse workflows, VM pools, and distributions, while handling per-type resource quotas. The approach demonstrates practical applicability for efficient, uncertainty-aware workflow scheduling in IaaS environments and suggests future work on automated task-model profiling and distribution fitting.

Abstract

The flexibility and the variety of computing resources offered by the cloud make it particularly attractive for executing user workloads. However, IaaS cloud environments pose non-trivial challenges in the case of workflow scheduling under deadlines and monetary cost constraints. Indeed, given the typical uncertain performance behavior of cloud resources, scheduling algorithms that assume deterministic execution times may fail, thus requiring probabilistic approaches. However, existing probabilistic algorithms are computationally expensive, mainly due to the greater complexity of the workflow scheduling problem in its probabilistic form, and they hardily scale with the size of the problem instance. In this article, we propose EPOSS, a novel workflow scheduling algorithm for IaaS cloud environments based on a probabilistic formulation. Our solution blends together the low execution latency of state-of-the-art scheduling algorithms designed for the case of deterministic execution times and the capability to enforce probabilistic constraints.Designed with computational efficiency in mind, EPOSS achieves one to two orders lower execution times in comparison with existing probabilistic schedulers. Furthermore, it ensures good scaling with respect to workflow size and number of heterogeneous virtual machine types offered by the IaaS cloud environment. We evaluated the benefits of our algorithm via an experimental comparison over a variety of workloads and characteristics of IaaS cloud environments.

Paper Structure

This paper contains 27 sections, 5 equations, 11 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Model and Problem Formulation
  4. Scheduling Algorithm
  5. MOHEFT
  6. EPOSS Design
  7. Modifications to MOHEFT
  8. EPOSS
  9. Parallel EPOSS
  10. Multi-objective Constrained Optimization Problem
  11. Experimental Evaluation
  12. Experimental Setting
  13. Workflows
  14. Task Execution Times
  15. VM Types
  16. ...and 12 more sections

Figures (11)

  • Figure 1: Graphical representations of the workflows used in the experimental study (figures taken from the Pegasus Workflow Repository: https://pegasus.isi.edu).
  • Figure 1: Graphical representation of the makespan, the monetary cost and the algorithm execution time with the CyberShake workflow ($p_T=0.9$)
  • Figure 2: Task execution times scalability scenarios A, B and C.
  • Figure 2: Graphical representation of the makespan, the monetary cost and the algorithm execution time with the LIGO workflow ($p_T=0.9$)
  • Figure 3: Aggregated results for different algorithms and different set of VM types
  • ...and 6 more figures