Reinforcement Learning based Workflow Scheduling in Cloud and Edge Computing Environments: A Taxonomy, Review and Future Directions

TL;DR

The paper tackles the challenge of scheduling workflows in dynamic cloud and edge environments using reinforcement learning and deep reinforcement learning. It introduces a comprehensive taxonomy spanning problem types, algorithms, objectives, architectures, training paradigms, and scheduling goals, and maps existing works to this framework to reveal strengths and gaps. By synthesizing a wide range of approaches—from single-agent to multi-agent systems and from SARSA/Q-learning to PPO and actor-critic methods—the authors highlight how DRL can address makespan, energy, cost, and SLA under highly dynamic conditions. The work underlines practical implications for auto-scaling, VM scheduling, and cross-layer orchestration while outlining future directions such as explicit MORL, CTDE-enabled MAS, accurate runtime estimation, asynchronous training, and scalable action spaces.

Abstract

Deep Reinforcement Learning (DRL) techniques have been successfully applied for solving complex decision-making and control tasks in multiple fields including robotics, autonomous driving, healthcare and natural language processing. The ability of DRL agents to learn from experience and utilize real-time data for making decisions makes it an ideal candidate for dealing with the complexities associated with the problem of workflow scheduling in highly dynamic cloud and edge computing environments. Despite the benefits of DRL, there are multiple challenges associated with the application of DRL techniques including multi-objectivity, curse of dimensionality, partial observability and multi-agent coordination. In this paper, we comprehensively analyze the challenges and opportunities associated with the design and implementation of DRL oriented solutions for workflow scheduling in cloud and edge computing environments. Based on the identified characteristics, we propose a taxonomy of workflow scheduling with DRL. We map reviewed works with respect to the taxonomy to identify their strengths and weaknesses. Based on taxonomy driven analysis, we propose novel future research directions for the field.

Figures (2)

• Figure 1: A potential architecture for a holistic workflow scheduling framework in a cloud datacenter with DRL (Actor-Critic method)
• Figure 2: Taxonomy of Workflow Scheduling in Cloud and Edge Computing Environments with Reinforcement Learning [CTE - Centralized Training and Execution, CTDE - Centralized Training and Distributed Execution, DTE - Distributed Training and Execution]