Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Digital Twin-Empowered Deep Reinforcement Learning for Intelligent VNF Migration in Edge-Core Networks

Faisal Ahmed, Suresh Subramaniam, Motoharu Matsuura, Hiroshi Hasegawa, Shih-Chun Lin

TL;DR

The paper addresses efficient VNF migration in edge–core networks under dynamic workloads by formulating migration as a Markov Decision Process and solving it with Advantage Actor‑Critic (A2C). It introduces a Digital Twin (DT) module comprising a multi‑task VAE and a multi‑task LSTM to generate synthetic experiences and predict environment dynamics, enabling faster policy convergence and lower real‑world exploration costs. Empirical results show the DT‑empowered DRL framework delivering substantial reductions in both average E2E delay and energy consumption, outperforming non‑DT baselines and random migration. The approach offers scalable, adaptive, and energy‑aware orchestration for heterogeneous edge–core infrastructures, with practical implications for low‑latency and green networking.

Abstract

The growing demand for services and the rapid deployment of virtualized network functions (VNFs) pose significant challenges for achieving low-latency and energy-efficient orchestration in modern edge-core network infrastructures. To address these challenges, this study proposes a Digital Twin (DT)-empowered Deep Reinforcement Learning framework for intelligent VNF migration that jointly minimizes average end-to-end (E2E) delay and energy consumption. By formulating the VNF migration problem as a Markov Decision Process and utilizing the Advantage Actor-Critic model, the proposed framework enables adaptive and real-time migration decisions. A key innovation of the proposed framework is the integration of a DT module composed of a multi-task Variational Autoencoder and a multi-task Long Short-Term Memory network. This combination collectively simulates environment dynamics and generates high-quality synthetic experiences, significantly enhancing training efficiency and accelerating policy convergence. Simulation results demonstrate substantial performance gains, such as significant reductions in both average E2E delay and energy consumption, thereby establishing new benchmarks for intelligent VNF migration in edge-core networks.

Digital Twin-Empowered Deep Reinforcement Learning for Intelligent VNF Migration in Edge-Core Networks

TL;DR

The paper addresses efficient VNF migration in edge–core networks under dynamic workloads by formulating migration as a Markov Decision Process and solving it with Advantage Actor‑Critic (A2C). It introduces a Digital Twin (DT) module comprising a multi‑task VAE and a multi‑task LSTM to generate synthetic experiences and predict environment dynamics, enabling faster policy convergence and lower real‑world exploration costs. Empirical results show the DT‑empowered DRL framework delivering substantial reductions in both average E2E delay and energy consumption, outperforming non‑DT baselines and random migration. The approach offers scalable, adaptive, and energy‑aware orchestration for heterogeneous edge–core infrastructures, with practical implications for low‑latency and green networking.

Abstract

The growing demand for services and the rapid deployment of virtualized network functions (VNFs) pose significant challenges for achieving low-latency and energy-efficient orchestration in modern edge-core network infrastructures. To address these challenges, this study proposes a Digital Twin (DT)-empowered Deep Reinforcement Learning framework for intelligent VNF migration that jointly minimizes average end-to-end (E2E) delay and energy consumption. By formulating the VNF migration problem as a Markov Decision Process and utilizing the Advantage Actor-Critic model, the proposed framework enables adaptive and real-time migration decisions. A key innovation of the proposed framework is the integration of a DT module composed of a multi-task Variational Autoencoder and a multi-task Long Short-Term Memory network. This combination collectively simulates environment dynamics and generates high-quality synthetic experiences, significantly enhancing training efficiency and accelerating policy convergence. Simulation results demonstrate substantial performance gains, such as significant reductions in both average E2E delay and energy consumption, thereby establishing new benchmarks for intelligent VNF migration in edge-core networks.

Paper Structure

This paper contains 15 sections, 26 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. VNF and Logical Link Mapping in VNF-FG, Associated Constraints, and Service Status
  4. E2E Delay Model
  5. Energy Consumption Model
  6. Problem Formulation
  7. DT-Empowered DRL for VNF Migration
  8. Key Procedures of the Proposed VNF Migration Scheme
  9. Description of the DRL Framework
  10. Description of the DT Module
  11. DT-Empowered DRL Framework Training
  12. Performance Evaluation
  13. Simulation Setup
  14. Numerical Results and Comparisons
  15. Conclusions

Figures (3)

  • Figure 1: A representative example of an 8K video streaming service served by a VNF-FG demonstrates that the E2E delay is reduced from 14 ms to 12 ms, and the edge server $s_{e}$ is powered off following VNF migration, thereby reducing overall energy consumption.
  • Figure 2: Operation of the DT-empowered DRL.
  • Figure 3: (a) Average energy consumption vs. number of VNF-FGs, (b) Average E2E delay vs. number of VNF-FGs, (c) Average request acceptance ratio vs. number of VNF-FGs.