Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Digital Twin-assisted Reinforcement Learning for Resource-aware Microservice Offloading in Edge Computing

Xiangchun Chen, Jiannong Cao, Zhixuan Liang, Yuvraj Sahni, Mingjin Zhang

TL;DR

This paper introduces a novel microservice offloading algorithm, DTDRLMO, which leverages deep reinforcement learning (DRL) and digital twin technology, and employs digital twin techniques to predict and adapt to changing edge node loads and network conditions of CEC in real-time.

Abstract

Collaborative edge computing (CEC) has emerged as a promising paradigm, enabling edge nodes to collaborate and execute microservices from end devices. Microservice offloading, a fundamentally important problem, decides when and where microservices are executed upon the arrival of services. However, the dynamic nature of the real-world CEC environment often leads to inefficient microservice offloading strategies, resulting in underutilized resources and network congestion. To address this challenge, we formulate an online joint microservice offloading and bandwidth allocation problem, JMOBA, to minimize the average completion time of services. In this paper, we introduce a novel microservice offloading algorithm, DTDRLMO, which leverages deep reinforcement learning (DRL) and digital twin technology. Specifically, we employ digital twin techniques to predict and adapt to changing edge node loads and network conditions of CEC in real-time. Furthermore, this approach enables the generation of an efficient offloading plan, selecting the most suitable edge node for each microservice. Simulation results on real-world and synthetic datasets demonstrate that DTDRLMO outperforms heuristic and learning-based methods in average service completion time.

Digital Twin-assisted Reinforcement Learning for Resource-aware Microservice Offloading in Edge Computing

TL;DR

This paper introduces a novel microservice offloading algorithm, DTDRLMO, which leverages deep reinforcement learning (DRL) and digital twin technology, and employs digital twin techniques to predict and adapt to changing edge node loads and network conditions of CEC in real-time.

Abstract

Collaborative edge computing (CEC) has emerged as a promising paradigm, enabling edge nodes to collaborate and execute microservices from end devices. Microservice offloading, a fundamentally important problem, decides when and where microservices are executed upon the arrival of services. However, the dynamic nature of the real-world CEC environment often leads to inefficient microservice offloading strategies, resulting in underutilized resources and network congestion. To address this challenge, we formulate an online joint microservice offloading and bandwidth allocation problem, JMOBA, to minimize the average completion time of services. In this paper, we introduce a novel microservice offloading algorithm, DTDRLMO, which leverages deep reinforcement learning (DRL) and digital twin technology. Specifically, we employ digital twin techniques to predict and adapt to changing edge node loads and network conditions of CEC in real-time. Furthermore, this approach enables the generation of an efficient offloading plan, selecting the most suitable edge node for each microservice. Simulation results on real-world and synthetic datasets demonstrate that DTDRLMO outperforms heuristic and learning-based methods in average service completion time.
Paper Structure (21 sections, 23 equations, 5 figures, 2 algorithms)

This paper contains 21 sections, 23 equations, 5 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Task Offloading in Edge Computing
  4. Microsevice Offloading in Edge Computing
  5. System Model and Problem Formulation
  6. System Model
  7. Microservice Model
  8. Digital twin-assisted Collaborative Scheduling
  9. Problem Formulation
  10. Digital Twin-assisted Deep Reinforcement Learning
  11. Markov Decision Process
  12. State Space
  13. Action Space
  14. Reward Function
  15. Digital Twin-assisted Deep Reinforcement Learning
  16. ...and 6 more sections

Figures (5)

  • Figure 1: The digital twin-based microservice offloading framework, consisting of the end device layer, physical CEC layer, and digital twin layer.
  • Figure 2: The diagram of digital twin-assisted deep deterministic policy gradient algorithm.
  • Figure 3: The values of reward, actor loss, and critic loss in the training process of the DTDRLMO.
  • Figure 4: Simulation Results for Real-world Dataset
  • Figure 5: Simulation Results for Synthetic Dataset