Cost-Aware Dynamic Cloud Workflow Scheduling using Self-Attention and Evolutionary Reinforcement Learning

TL;DR

A novel self-attention policy network for cloud workflow scheduling (SPN-CWS) that captures global information from all VMs is proposed and an Evolution Strategy-based RL (ERL) system is developed to train SPN-CWS reliably and effectively.

Abstract

The Cost-aware Dynamic Multi-Workflow Scheduling (CDMWS) in the cloud is a kind of cloud workflow management problem, which aims to assign virtual machine (VM) instances to execute tasks in workflows so as to minimize the total costs, including both the penalties for violating Service Level Agreement (SLA) and the VM rental fees. Powered by deep neural networks, Reinforcement Learning (RL) methods can construct effective scheduling policies for solving CDMWS problems. Traditional policy networks in RL often use basic feedforward architectures to separately determine the suitability of assigning any VM instances, without considering all VMs simultaneously to learn their global information. This paper proposes a novel self-attention policy network for cloud workflow scheduling (SPN-CWS) that captures global information from all VMs. We also develop an Evolution Strategy-based RL (ERL) system to train SPN-CWS reliably and effectively. The trained SPN-CWS can effectively process all candidate VM instances simultaneously to identify the most suitable VM instance to execute every workflow task. Comprehensive experiments show that our method can noticeably outperform several state-of-the-art algorithms on multiple benchmark CDMWS problems.

Figures (5)

• Figure 1: The diagram of the broker performing the workflow scheduling.
• Figure 2: The diagram of the scheduling of CDMWS.
• Figure 3: The structure of SPN-CWS.
• Figure 4: The average of VM fees and SLA penalties of all algorithms.
• Figure 5: The convergence on small-scenario CDMWS instance with $\gamma=5$.