Joint Service Caching, Communication and Computing Resource Allocation in Collaborative MEC Systems: A DRL-based Two-timescale Approach

TL;DR

This paper tackles the problem of joint service caching, communication and computing resource allocation in collaborative MEC systems subject to two timescales. It introduces DGL-DDPG, a two-layer DRL framework that uses LSTM-DDPG to determine long-term service caching at large timescales and an Improved-GA to make instantaneous Off-RA decisions at small timescales, linking the layers via an MD-like formulation. The approach demonstrates superior long-term QoS and reduced cache switching costs compared with baselines such as GA, DDPG, and popularity-based caching, while benefiting from collaboration among edge servers. The framework offers a practical path toward reducing backhaul and caching overhead, improving resource utilization, and supporting delay-sensitive IIoT applications in MEC environments.

Abstract

Meeting the strict Quality of Service (QoS) requirements of terminals has imposed a signiffcant challenge on Multiaccess Edge Computing (MEC) systems, due to the limited multidimensional resources. To address this challenge, we propose a collaborative MEC framework that facilitates resource sharing between the edge servers, and with the aim to maximize the long-term QoS and reduce the cache switching cost through joint optimization of service caching, collaborative offfoading, and computation and communication resource allocation. The dual timescale feature and temporal recurrence relationship between service caching and other resource allocation make solving the problem even more challenging. To solve it, we propose a deep reinforcement learning (DRL)-based dual timescale scheme, called DGL-DDPG, which is composed of a short-term genetic algorithm (GA) and a long short-term memory network-based deep deterministic policy gradient (LSTM-DDPG). In doing so, we reformulate the optimization problem as a Markov decision process (MDP) where the small-timescale resource allocation decisions generated by an improved GA are taken as the states and input into a centralized LSTM-DDPG agent to generate the service caching decision for the large-timescale. Simulation results demonstrate that our proposed algorithm outperforms the baseline algorithms in terms of the average QoS and cache switching cost.

Figures (10)

• Figure 1: Illustration of service caching, communication and computing in a collaborative MEC-enabled IIoT system.
• Figure 2: The framework of the DGL-DDPG.
• Figure 3: The framework of the small-timescale Improved GA based Off-RA.
• Figure 4: Convergence performance of Improved-GA.
• Figure 5: Convergence performance of accumulative reward for LSTM-DDPG.