Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Neural Network Aided Deep Reinforcement Learning for Resource Allocation in Dynamic Terahertz UAV Networks

Zhifeng Hu, Chong Han

TL;DR

This work tackles the challenging problem of long-term resource allocation in dynamic THz UAV networks by formulating a MINLP for joint power and sub-array usage and solving it with a graph neural network aided DRL framework, GLOVE. By combining GCN-based neighbor awareness with emphasis on self-node features through FC pathways, GLOVE learns a cooperative, continuous-action policy via DDPG, optimized for resource efficiency, latency, and packet loss. Empirical results show GLOVE achieves the highest RE and the lowest latency with zero packet loss, significantly outperforming baseline GNN-DRL and multi-agent methods, while maintaining favorable computational and memory efficiency. This approach offers a scalable, robust solution for real-time resource management in highly dynamic THz UAV networks, enabling ultra-high-rate, reliable inter-UAV communication and ISAC-enabled routing to LEO gateways.

Abstract

Terahertz (THz) unmanned aerial vehicle (UAV) networks with flexible topologies and ultra-high data rates are expected to empower numerous applications in security surveillance, disaster response, and environmental monitoring, among others. However, the dynamic topologies hinder the efficient long-term joint power and antenna array resource allocation for THz links among UAVs. Furthermore, the continuous nature of power and the discrete nature of antennas cause this joint resource allocation problem to be a mixed-integer nonlinear programming (MINLP) problem with non-convexity and NP-hardness. Inspired by recent rapid advancements in deep reinforcement learning (DRL), a graph neural network (GNN) aided DRL algorithm for resource allocation in the dynamic THz UAV network with an emphasis on self-node features (GLOVE) is proposed in this paper, with the aim of resource efficiency (RE) maximization. When training the allocation policy for each UAV, GLOVE learns the relationship between this UAV and its neighboring UAVs via GNN, while also emphasizing the important self-node features of this UAV. In addition, a multi-task structure is leveraged by GLOVE to cooperatively train resource allocation decisions for the power and sub-arrays of all UAVs. Experimental results illustrate that GLOVE outperforms benchmark schemes in terms of the highest RE and the lowest latency. Moreover, unlike the benchmark methods with severe packet loss, GLOVE maintains zero packet loss during the entire training process, demonstrating its better robustness under the highly dynamic THz UAV network.

Graph Neural Network Aided Deep Reinforcement Learning for Resource Allocation in Dynamic Terahertz UAV Networks

TL;DR

This work tackles the challenging problem of long-term resource allocation in dynamic THz UAV networks by formulating a MINLP for joint power and sub-array usage and solving it with a graph neural network aided DRL framework, GLOVE. By combining GCN-based neighbor awareness with emphasis on self-node features through FC pathways, GLOVE learns a cooperative, continuous-action policy via DDPG, optimized for resource efficiency, latency, and packet loss. Empirical results show GLOVE achieves the highest RE and the lowest latency with zero packet loss, significantly outperforming baseline GNN-DRL and multi-agent methods, while maintaining favorable computational and memory efficiency. This approach offers a scalable, robust solution for real-time resource management in highly dynamic THz UAV networks, enabling ultra-high-rate, reliable inter-UAV communication and ISAC-enabled routing to LEO gateways.

Abstract

Terahertz (THz) unmanned aerial vehicle (UAV) networks with flexible topologies and ultra-high data rates are expected to empower numerous applications in security surveillance, disaster response, and environmental monitoring, among others. However, the dynamic topologies hinder the efficient long-term joint power and antenna array resource allocation for THz links among UAVs. Furthermore, the continuous nature of power and the discrete nature of antennas cause this joint resource allocation problem to be a mixed-integer nonlinear programming (MINLP) problem with non-convexity and NP-hardness. Inspired by recent rapid advancements in deep reinforcement learning (DRL), a graph neural network (GNN) aided DRL algorithm for resource allocation in the dynamic THz UAV network with an emphasis on self-node features (GLOVE) is proposed in this paper, with the aim of resource efficiency (RE) maximization. When training the allocation policy for each UAV, GLOVE learns the relationship between this UAV and its neighboring UAVs via GNN, while also emphasizing the important self-node features of this UAV. In addition, a multi-task structure is leveraged by GLOVE to cooperatively train resource allocation decisions for the power and sub-arrays of all UAVs. Experimental results illustrate that GLOVE outperforms benchmark schemes in terms of the highest RE and the lowest latency. Moreover, unlike the benchmark methods with severe packet loss, GLOVE maintains zero packet loss during the entire training process, demonstrating its better robustness under the highly dynamic THz UAV network.
Paper Structure (17 sections, 22 equations, 10 figures, 1 table, 1 algorithm)

This paper contains 17 sections, 22 equations, 10 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. THz UAV Network Model
  4. THz Transmission Model
  5. Joint Power and Sub-array Allocation Problem Formulation
  6. Deep Reinforcement Learning Framework
  7. State
  8. Action
  9. Reward
  10. Graph Neural Network Aided Deep Reinforcement Learning for Resource Allocation with an Emphasis on Self-node Features
  11. Overall Learning Framework
  12. GNN-aided Actor and Critic Networks with an Emphasis on Self-node Features
  13. GLOVE Algorithm
  14. Simulation Results and Analysis
  15. Simulation Parameters of THz UAV Network
  16. ...and 2 more sections

Figures (10)

  • Figure 1: THz UAV network.
  • Figure 2: Allocation and learning process of GLOVE.
  • Figure 3: GLOVE agent architectures (a) of the actor network, and (b) of the critic network.
  • Figure 4: THz UAV network topologies (a) at 0 s, (b) at 10 s, and (c) at 20 s.
  • Figure 5: Resource usage ratio comparison.
  • ...and 5 more figures