Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Digital Twin-Assisted Data-Driven Optimization for Reliable Edge Caching in Wireless Networks

Zifan Zhang, Yuchen Liu, Zhiyuan Peng, Mingzhe Chen, Dongkuan Xu, Shuguang Cui

TL;DR

This work tackles reliable edge caching in dense nextG networks by integrating digital twins (DTs) with constrained Markov decision process–based reinforcement learning. The proposed D-REC framework uses a three-stage network DT creation/synchronization process (dynamic connectivity segmentation, vertical twinning, horizontal twinning) to generate rich data for training and safety evaluation, while embedding four reliability intervention modules that modify state, action, or reward to prevent BS overloads and ensure load balance. The authors prove that these reliability additions do not degrade convergence and demonstrate through simulations and real-data traces that D-REC improves cache hit rates and load balancing compared with traditional caching methods and basic DRL baselines. The DT-enabled, reliability-aware approach enhances robustness and predictability of caching decisions in nextG wireless networks, supporting scalable, low-latency content delivery in highly dynamic environments. The framework has practical implications for deploying reliable, data-driven caching policies in real-world edge networks and offers a pathway to safer, more stable RL-based networking solutions, even under distributional uncertainty such as Zipf-like content popularity.

Abstract

Optimizing edge caching is crucial for the advancement of next-generation (nextG) wireless networks, ensuring high-speed and low-latency services for mobile users. Existing data-driven optimization approaches often lack awareness of the distribution of random data variables and focus solely on optimizing cache hit rates, neglecting potential reliability concerns, such as base station overload and unbalanced cache issues. This oversight can result in system crashes and degraded user experience. To bridge this gap, we introduce a novel digital twin-assisted optimization framework, called D-REC, which integrates reinforcement learning (RL) with diverse intervention modules to ensure reliable caching in nextG wireless networks. We first develop a joint vertical and horizontal twinning approach to efficiently create network digital twins, which are then employed by D-REC as RL optimizers and safeguards, providing ample datasets for training and predictive evaluation of our cache replacement policy. By incorporating reliability modules into a constrained Markov decision process, D-REC can adaptively adjust actions, rewards, and states to comply with advantageous constraints, minimizing the risk of network failures. Theoretical analysis demonstrates comparable convergence rates between D-REC and vanilla data-driven methods without compromising caching performance. Extensive experiments validate that D-REC outperforms conventional approaches in cache hit rate and load balancing while effectively enforcing predetermined reliability intervention modules.

Digital Twin-Assisted Data-Driven Optimization for Reliable Edge Caching in Wireless Networks

TL;DR

This work tackles reliable edge caching in dense nextG networks by integrating digital twins (DTs) with constrained Markov decision process–based reinforcement learning. The proposed D-REC framework uses a three-stage network DT creation/synchronization process (dynamic connectivity segmentation, vertical twinning, horizontal twinning) to generate rich data for training and safety evaluation, while embedding four reliability intervention modules that modify state, action, or reward to prevent BS overloads and ensure load balance. The authors prove that these reliability additions do not degrade convergence and demonstrate through simulations and real-data traces that D-REC improves cache hit rates and load balancing compared with traditional caching methods and basic DRL baselines. The DT-enabled, reliability-aware approach enhances robustness and predictability of caching decisions in nextG wireless networks, supporting scalable, low-latency content delivery in highly dynamic environments. The framework has practical implications for deploying reliable, data-driven caching policies in real-world edge networks and offers a pathway to safer, more stable RL-based networking solutions, even under distributional uncertainty such as Zipf-like content popularity.

Abstract

Optimizing edge caching is crucial for the advancement of next-generation (nextG) wireless networks, ensuring high-speed and low-latency services for mobile users. Existing data-driven optimization approaches often lack awareness of the distribution of random data variables and focus solely on optimizing cache hit rates, neglecting potential reliability concerns, such as base station overload and unbalanced cache issues. This oversight can result in system crashes and degraded user experience. To bridge this gap, we introduce a novel digital twin-assisted optimization framework, called D-REC, which integrates reinforcement learning (RL) with diverse intervention modules to ensure reliable caching in nextG wireless networks. We first develop a joint vertical and horizontal twinning approach to efficiently create network digital twins, which are then employed by D-REC as RL optimizers and safeguards, providing ample datasets for training and predictive evaluation of our cache replacement policy. By incorporating reliability modules into a constrained Markov decision process, D-REC can adaptively adjust actions, rewards, and states to comply with advantageous constraints, minimizing the risk of network failures. Theoretical analysis demonstrates comparable convergence rates between D-REC and vanilla data-driven methods without compromising caching performance. Extensive experiments validate that D-REC outperforms conventional approaches in cache hit rate and load balancing while effectively enforcing predetermined reliability intervention modules.
Paper Structure (32 sections, 1 theorem, 21 equations, 13 figures, 2 tables, 3 algorithms)

This paper contains 32 sections, 1 theorem, 21 equations, 13 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Work
  3. Cache Replacement and Problem Formulation
  4. Related Work on Edge Caching
  5. Reliable Reinforcement Learning as a Primer
  6. Digital Twin as Data Generator for What-if Analysis
  7. Creation and Synchronization of Network Digital Twins
  8. Dynamic Connectivity Segmentation
  9. Vertical Twinning for Initialization
  10. Horizontal Twinning for Evolution
  11. Caching Optimization with Constrained Markov Decision Process
  12. Problem Formulation with CMDP Model
  13. RL-based Solution
  14. D-REC: Reliability Module Integration
  15. REC with Network Digital Twin (D-REC)
  16. ...and 17 more sections

Key Result

Theorem 1

Let $F$ and $G$ be given in Eq. (21)-(22), with $\{ H_j\}_{j\in[q]}$ being absolute constants. For any $j \in [q-1]$, we define $\beta_j^* = \beta_j \cdot \prod_{\ell = j+1} \min ( \beta_{\ell}, 1)$, $\beta_q^* = 1$, and $\alpha^* = \max_{ j \in [q] } t_j / (2 \beta_j^* + t_j )$. For the parameters For any $K \in \mathbb{N}$, let $Q^{\pi_K}$ be the action-value function corresponding to policy $\

Figures (13)

  • Figure 1: Overall caching optimization framework in cellular networks.
  • Figure 2: Example of a cache replacement problem.
  • Figure 3: A zoom-in view of intervention modules. The grey box indicates the effective area of reliability intervention modules.
  • Figure 4: Cache hit rate when incorporating reliability intervention modules.
  • Figure 5: Unreliable action mutation count with different reliability modules. Note that the pure D-REC and D-REC with the reward intervention module do not perform any action mutations, resulting in a count that remains at 0.
  • ...and 8 more figures

Theorems & Definitions (1)

  • Theorem 1