Multi-Objective Offloading Optimization in MEC and Vehicular-Fog Systems: A Distributed-TD3 Approach

TL;DR

This work tackles computational offloading in a two-tier MEC and vehicular-fog network to mitigate MEC congestion during hotspot events. It introduces a Distributed-TD3 reinforcement learning framework with multiple actor networks and two critics to learn joint offloading decisions (local, horizontal, vertical) that minimize a weighted latency-energy cost $C_{sys} = \min(\sigma L_{sys} + (1-\sigma) E_{sys})$. Through OpenAI Gym-based simulations, the proposed method demonstrates faster convergence and superior latency-energy performance compared with TD3, DDPG, SA, and PSO baselines, particularly under high arrival traffic and substantial VF resources. The approach provides a practical control-plane mechanism for dynamic load balancing and efficient resource utilization in MEC-VF ecosystems, with implications for scalable, low-latency IoT services.

Abstract

The emergence of 5G networks has enabled the deployment of a two-tier edge and vehicular-fog network. It comprises Multi-access Edge Computing (MEC) and Vehicular-Fogs (VFs), strategically positioned closer to Internet of Things (IoT) devices, reducing propagation latency compared to cloud-based solutions and ensuring satisfactory quality of service (QoS). However, during high-traffic events like concerts or athletic contests, MEC sites may face congestion and become overloaded. Utilizing offloading techniques, we can transfer computationally intensive tasks from resource-constrained devices to those with sufficient capacity, for accelerating tasks and extending device battery life. In this research, we consider offloading within a two-tier MEC and VF architecture, involving offloading from MEC to MEC and from MEC to VF. The primary objective is to minimize the average system cost, considering both latency and energy consumption. To achieve this goal, we formulate a multi-objective optimization problem aimed at minimizing latency and energy while considering given resource constraints. To facilitate decision-making for nearly optimal computational offloading, we design an equivalent reinforcement learning environment that accurately represents the network architecture and the formulated problem. To accomplish this, we propose a Distributed-TD3 (DTD3) approach, which builds on the TD3 algorithm. Extensive simulations, demonstrate that our strategy achieves faster convergence and higher efficiency compared to other benchmark solutions.

Figures (8)

• Figure 1: MEC and vehicular-fog architecture.
• Figure 2: Distributed-TD3 model for MEC and VF architecture.
• Figure 3: Performance with change in weighting parameter ($\sigma$).
• Figure 4: DTD3 model performance in a single run.
• Figure 5: DTD3 model performance with different arrival traffic.