DeepEdge: A Deep Reinforcement Learning based Task Orchestrator for Edge Computing

TL;DR

This paper tackles the challenge of task offloading in dynamic, multi-application edge computing environments by formulating it as an online reinforcement learning problem. It introduces DeepEdge, a Double Deep Q-Network (DDQN)–based task orchestrator that learns to route tasks to edge servers or the cloud while accounting for mobility, heterogeneous applications, and network delays. The authors implement a detailed MDP framework with a robust state representation, a closed-form reward structure, and a mechanism to handle delayed action effects, and they validate the approach in EdgeCloudSim against heuristic baselines, showing superior reliability under heavy load and better resource utilization. The work demonstrates the practicality of online DRL for adaptive edge orchestration and suggests future directions like federated learning to cope with diverse task arrival rates across applications.

Abstract

The improvements in the edge computing technology pave the road for diversified applications that demand real-time interaction. However, due to the mobility of the end-users and the dynamic edge environment, it becomes challenging to handle the task offloading with high performance. Moreover, since each application in mobile devices has different characteristics, a task orchestrator must be adaptive and have the ability to learn the dynamics of the environment. For this purpose, we develop a deep reinforcement learning based task orchestrator, DeepEdge, which learns to meet different task requirements without needing human interaction even under the heavily-loaded stochastic network conditions in terms of mobile users and applications. Given the dynamic offloading requests and time-varying communication conditions, we successfully model the problem as a Markov process and then apply the Double Deep Q-Network (DDQN) algorithm to implement DeepEdge. To evaluate the robustness of DeepEdge, we experiment with four different applications including image rendering, infotainment, pervasive health, and augmented reality in the network under various loads. Furthermore, we compare the performance of our agent with the four different task offloading approaches in the literature. Our results show that DeepEdge outperforms its competitors in terms of the percentage of satisfactorily completed tasks.

Figures (12)

• Figure 1: A three-tier edge computing network.
• Figure 2: Because of the characteristics of the edge computing environment, two consecutive states would be the same regarding the application and network attributes that define the states. This is impractical for DRL since the Markov property may not be ensured. Therefore, we use four supplemental attributes for each state to identify them uniquely.
• Figure 3: The state transition in an edge computing environment consisting of two edge servers.
• Figure 4: The generic deep neural network model for DeepEdge.
• Figure 5: 5-tuple memory item structure to feed the DDQN algorithm.