Handover and SINR-Aware Path Optimization in 5G-UAV mmWave Communication using DRL

TL;DR

The paper addresses UAV trajectory optimization in 5G mmWave networks under dynamic SINR and handover conditions. It proposes a model-free actor-critic DRL approach that jointly optimizes flight time, handovers, connectivity, and SINR, trained with Wireless InSite ray-tracing data. Results show the AC-DRL method delivers higher SINR and robust trajectories compared with a DDQN baseline, validating its effectiveness in realistic urban channels. By leveraging ray-tracing data, the work demonstrates a practical path toward improved UAV mobility management and connectivity in dense 5G deployments.

Abstract

Path planning and optimization for unmanned aerial vehicles (UAVs)-assisted next-generation wireless networks is critical for mobility management and ensuring UAV safety and ubiquitous connectivity, especially in dense urban environments with street canyons and tall buildings. Traditional statistical and model-based techniques have been successfully used for path optimization in communication networks. However, when dynamic channel propagation characteristics such as line-of-sight (LOS), interference, handover, and signal-to-interference and noise ratio (SINR) are included in path optimization, statistical and model-based path planning solutions become obsolete since they cannot adapt to the dynamic and time-varying wireless channels, especially in the mmWave bands. In this paper, we propose a novel model-free actor-critic deep reinforcement learning (AC-DRL) framework for path optimization in UAV-assisted 5G mmWave wireless networks, which combines four important aspects of UAV communication: \textit{flight time, handover, connectivity and SINR}. We train an AC-RL agent that enables a UAV connected to a gNB to determine the optimal path to a desired destination in the shortest possible time with minimal gNB handover, while maintaining connectivity and the highest possible SINR. We train our model with data from a powerful ray tracing tool called Wireless InSite, which uses 3D images of the propagation environment and provides data that closely resembles the real propagation environment. The simulation results show that our system has superior performance in tracking high SINR compared to other selected RL algorithms.

Figures (8)

• Figure 1: Pisa City
• Figure 2: Reference Scenario: A grid of the UAV navigation environment on a horizontal plane 150m above Pisa City.
• Figure 3: System Architecture
• Figure 4: A sample of Wireless InSite ray propagation. The green marks show the gNB locations. Notice the reflections and diffractions at surfaces and edges.
• Figure 5: UAV-Trajectories using the trained models (green=AC, magenta=DDQN). The coloured grid shows the maximum SINR at each UAV position in the navigation environment.