EdgeTimer: Adaptive Multi-Timescale Scheduling in Mobile Edge Computing with Deep Reinforcement Learning

TL;DR

EdgeTimer tackles adaptive, multi-layer scheduling in mobile edge computing by learning layer-specific update timescales with a three-layer hierarchical DRL. It couples HDRL with a safe, decentralized MADRL framework (Dec-POMDP) to decide when to update edge-cloud placement, edge-edge offloading, and intra-edge allocation, while masking unsafe actions. Empirical evaluation on a Vienna-based MEC simulator with 45 built-in rules and Alibaba workload traces shows EdgeTimer can boost profit by up to $9.1\times$ without harming delay, and converges efficiently (offline ~35 minutes) with fast online inference (~$0.026$ s). The work contributes a novel adaptive, asynchronous, and autonomous scheduling paradigm that can plug into existing MEC rules and scale across diverse workload patterns, offering practical gains for service providers.

Abstract

In mobile edge computing (MEC), resource scheduling is crucial to task requests' performance and service providers' cost, involving multi-layer heterogeneous scheduling decisions. Existing schedulers typically adopt static timescales to regularly update scheduling decisions of each layer, without adaptive adjustment of timescales for different layers, resulting in potentially poor performance in practice. We notice that the adaptive timescales would significantly improve the trade-off between the operation cost and delay performance. Based on this insight, we propose EdgeTimer, the first work to automatically generate adaptive timescales to update multi-layer scheduling decisions using deep reinforcement learning (DRL). First, EdgeTimer uses a three-layer hierarchical DRL framework to decouple the multi-layer decision-making task into a hierarchy of independent sub-tasks for improving learning efficiency. Second, to cope with each sub-task, EdgeTimer adopts a safe multi-agent DRL algorithm for decentralized scheduling while ensuring system reliability. We apply EdgeTimer to a wide range of Kubernetes scheduling rules, and evaluate it using production traces with different workload patterns. Extensive trace-driven experiments demonstrate that EdgeTimer can learn adaptive timescales, irrespective of workload patterns and built-in scheduling rules. It obtains up to 9.1x more profit than existing approaches without sacrificing the delay performance.

Figures (11)

• Figure 1: Timescales of different resource scheduling frameworks in MEC. The red dotted line shows the asynchrony nature of the adaptive multi-timescale framework, i.e., EdgeTimer, where the higher-layer scheduling decision is updated but lower-layer decisions remain unchanged.
• Figure 2: An example to show that by cumulatively applying EdgeTimer's features of adaptation (F1), asynchrony (F2) and autonomy (F3) to existing approaches, the profit can be improved significantly. The scheduling decisions are updated at the moment of the red vertical lines.
• Figure 3: The illustration of three-layer heterogeneous scheduling in MEC.
• Figure 4: The workflow of three-layer hierarchical DRL framework.
• Figure 5: The illustration of safe multi-agent DRL algorithm.