Multi-Agent DRL for Multi-Objective Twin Migration Routing with Workload Prediction in 6G-enabled IoV

TL;DR

The paper tackles VT migration in 6G-enabled IoV under dynamic vehicle mobility and uneven edge-server distribution by formulating a multi-objective optimization and proposing a workload-prediction driven framework. It combines an LSTM-enhanced Transformer model to forecast edge workloads with a DM-MAPPO multi-agent reinforcement learning approach that uses a dynamic mask to prune infeasible actions and discover multiple efficient VT-migration routes. Empirical results show significant improvements in latency ($O_T$ reduced), packet loss ($O_D$ reduced), and workload balance ($O_V$ reduced) when compared to baselines, with the LSTM-based predictions enabling the DM-MAPPO to avoid infeasible migrations and to coordinate decisions among agents. The framework leverages RSUs, UAVs, and satellites to maintain VT service continuity in diverse regions, offering practical impact for 6G IoV deployments and real-time vehicle applications, and it includes a privacy-preserving extension as future work.

Abstract

Sixth Generation (6G)-enabled Internet of Vehicles (IoV) facilitates efficient data synchronization through ultra-fast bandwidth and high-density connectivity, enabling the emergence of Vehicle Twins (VTs). As highly accurate replicas of vehicles, VTs can support intelligent vehicular applications for occupants in 6G-enabled IoV. Thanks to the full coverage capability of 6G, resource-constrained vehicles can offload VTs to edge servers, such as roadside units, unmanned aerial vehicles, and satellites, utilizing their computing and storage resources for VT construction and updates. However, communication between vehicles and edge servers with limited coverage is prone to interruptions due to the dynamic mobility of vehicles. Consequently, VTs must be migrated among edge servers to maintain uninterrupted and high-quality services for users. In this paper, we introduce a VT migration framework in 6G-enabled IoV. Specifically, we first propose a Long Short-Term Memory (LSTM)-based Transformer model to accurately predict long-term workloads of edge servers for migration decision-making. Then, we propose a Dynamic Mask Multi-Agent Proximal Policy Optimization (DM-MAPPO) algorithm to identify optimal migration routes in the highly complex environment of 6G-enabled IoV. Finally, we develop a practical platform to validate the effectiveness of the proposed scheme using real datasets. Simulation results demonstrate that the proposed DM-MAPPO algorithm significantly reduces migration latency by 20.82% and packet loss by 75.07% compared with traditional deep reinforcement learning algorithms.

Figures (10)

• Figure 1: The workload prediction-based VT migration framework in 6G-enabled IoV networks.
• Figure 2: The structure of the LSTM-based Transformer model.
• Figure 3: The algorithm architecture of DM-MAPPO for optimal VT migration routes.
• Figure 5: The workload of edge servers predicted by different prediction algorithms over time.
• Figure 6: Performance comparison of test reward curves for various DRL algorithms under the dynamic mask module.