Deep Graph Reinforcement Learning for UAV-Enabled Multi-User Secure Communications

TL;DR

The paper tackles physical-layer security in UAV-enabled multi-user networks under high dynamics by proposing a hierarchical deep learning framework that combines graph neural networks for secure beamforming with soft actor-critic reinforcement learning for UAV deployment. The inner GNN handles interference-aware beamforming as a graph problem with permutation-equivalent structure, while the outer SAC module optimizes UAV placement using rewards computed from the GNN-enhanced secrecy rate. Key contributions include a permutation-equivariant GNN architecture for scalable beamforming, unsupervised training of beamformers, and a SAC-based deployment strategy that adapts to changing networks, achieving near-optimal secrecy with significantly reduced computation time. The approach demonstrates strong generalization to unseen network sizes and configurations, offering a practical, scalable solution for secure UAV communications in dynamic environments.

Abstract

While unmanned aerial vehicles (UAVs) with flexible mobility are envisioned to enhance physical layer security in wireless communications, the efficient security design that adapts to such high network dynamics is rather challenging. The conventional approaches extended from optimization perspectives are usually quite involved, especially when jointly considering factors in different scales such as deployment and transmission in UAV-related scenarios. In this paper, we address the UAV-enabled multi-user secure communications by proposing a deep graph reinforcement learning framework. Specifically, we reinterpret the security beamforming as a graph neural network (GNN) learning task, where mutual interference among users is managed through the message-passing mechanism. Then, the UAV deployment is obtained through soft actor-critic reinforcement learning, where the GNN-based security beamforming is exploited to guide the deployment strategy update. Simulation results demonstrate that the proposed approach achieves near-optimal security performance and significantly enhances the efficiency of strategy determination. Moreover, the deep graph reinforcement learning framework offers a scalable solution, adaptable to various network scenarios and configurations, establishing a robust basis for information security in UAV-enabled communications.

Figures (11)

• Figure 1: System model.
• Figure 2: GNN architecture for security beamforming.
• Figure 3: SAC architecture for UAV deployment.
• Figure 4: Convergence of GNN model training.
• Figure 5: Convergence of GNN model under changing scenarios.