Joint Optimization of UAV-Carried IRS for Urban Low Altitude mmWave Communications with Deep Reinforcement Learning

TL;DR

The paper tackles robust urban low-altitude mmWave communications by deploying a UAV-carried IRS to rebuild blocked links. It formulates a non-convex, dynamic joint optimization of IRS phase shifts and UAV trajectory and introduces EPPO, an enhanced DRL framework combining neural episodic control, MogLSTM, and a phase-shift strategy to accelerate learning and stabilize convergence. Through extensive simulations in single- and multi-user urban scenarios, EPPO outperforms DDPG, TD3, SAC, and PPO in cumulative reward, data rate, and UAV energy efficiency, while Jain fairness improves balance among users. The approach demonstrates practical viability for real-time adaptive IRS-assisted UAV networks, with policy deployment feasible via onboard FPGA and TDMA-based signaling. The work advances online, high-performance optimization for AIRS-enabled mmWave networks and provides a blueprint for future IRS-UAV designs in dense urban environments.

Abstract

Emerging technologies in sixth generation (6G) of wireless communications, such as terahertz communication and ultra-massive multiple-input multiple-output, present promising prospects. Despite the high data rate potential of millimeter wave communications, millimeter wave (mmWave) communications in urban low altitude economy (LAE) environments are constrained by challenges such as signal attenuation and multipath interference. Specially, in urban environments, mmWave communication experiences significant attenuation due to buildings, owing to its short wavelength, which necessitates developing innovative approaches to improve the robustness of such communications in LAE networking. In this paper, we explore the use of an unmanned aerial vehicle (UAV)-carried intelligent reflecting surface (IRS) to support low altitude mmWave communication. Specifically, we consider a typical urban low altitude communication scenario where a UAV-carried IRS establishes a line-of-sight (LoS) channel between the mobile users and a source user (SU) despite the presence of obstacles. Subsequently, we formulate an optimization problem aimed at maximizing the transmission rates and minimizing the energy consumption of the UAV by jointly optimizing phase shifts of the IRS and UAV trajectory. Given the non-convex nature of the problem and its high dynamics, we propose a deep reinforcement learning-based approach incorporating neural episodic control, long short-term memory, and an IRS phase shift control method to enhance the stability and accelerate the convergence. Simulation results show that the proposed algorithm effectively resolves the problem and surpasses other benchmark algorithms in various performances.

Figures (12)

• Figure 1: UAV-carried IRS-aided mmWave communication system in urban low altitude communication scenarios.
• Figure 2: The overview of NECSA.
• Figure 3: Mogrifier LSTM with 5 rounds of updates.
• Figure 4: The framework of the the proposed EPPO.
• Figure 5: Training results in the single-user scenario. (a) Cumulative rewards training curve. (b) Average rate training curve. (c) Cumulative energy consumption training curve.