Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Novel Joint DRL-Based Utility Optimization for UAV Data Services

Xuli Cai, Poonam Lohan, Burak Kantarci

TL;DR

This work tackles maximizing the number of served ground users $N_s$ in a UAV-assisted network under total bandwidth $B_t$ and total power $P_t$. It introduces a joint DRL framework combining discrete-action DQN for bandwidth block allocation with continuous-action DDPG for power control, integrated with a practical air-ground channel model featuring LoS with Rician fading and NLoS with Rayleigh fading. The approach demonstrates up to a $41\%$ improvement in served users over equal-resource allocation and provides insights into the effects of UAV altitude and fading on performance. This data-driven resource-management method offers scalable applicability to UAV networks with heterogeneous data-rate demands in dynamic environments, with potential implications for emergency and remote-area communications.

Abstract

In this paper, we propose a novel joint deep reinforcement learning (DRL)-based solution to optimize the utility of an uncrewed aerial vehicle (UAV)-assisted communication network. To maximize the number of users served within the constraints of the UAV's limited bandwidth and power resources, we employ deep Q-Networks (DQN) and deep deterministic policy gradient (DDPG) algorithms for optimal resource allocation to ground users with heterogeneous data rate demands. The DQN algorithm dynamically allocates multiple bandwidth resource blocks to different users based on current demand and available resource states. Simultaneously, the DDPG algorithm manages power allocation, continuously adjusting power levels to adapt to varying distances and fading conditions, including Rayleigh fading for non-line-of-sight (NLoS) links and Rician fading for line-of-sight (LoS) links. Our joint DRL-based solution demonstrates an increase of up to 41% in the number of users served compared to scenarios with equal bandwidth and power allocation.

A Novel Joint DRL-Based Utility Optimization for UAV Data Services

TL;DR

This work tackles maximizing the number of served ground users in a UAV-assisted network under total bandwidth and total power . It introduces a joint DRL framework combining discrete-action DQN for bandwidth block allocation with continuous-action DDPG for power control, integrated with a practical air-ground channel model featuring LoS with Rician fading and NLoS with Rayleigh fading. The approach demonstrates up to a improvement in served users over equal-resource allocation and provides insights into the effects of UAV altitude and fading on performance. This data-driven resource-management method offers scalable applicability to UAV networks with heterogeneous data-rate demands in dynamic environments, with potential implications for emergency and remote-area communications.

Abstract

In this paper, we propose a novel joint deep reinforcement learning (DRL)-based solution to optimize the utility of an uncrewed aerial vehicle (UAV)-assisted communication network. To maximize the number of users served within the constraints of the UAV's limited bandwidth and power resources, we employ deep Q-Networks (DQN) and deep deterministic policy gradient (DDPG) algorithms for optimal resource allocation to ground users with heterogeneous data rate demands. The DQN algorithm dynamically allocates multiple bandwidth resource blocks to different users based on current demand and available resource states. Simultaneously, the DDPG algorithm manages power allocation, continuously adjusting power levels to adapt to varying distances and fading conditions, including Rayleigh fading for non-line-of-sight (NLoS) links and Rician fading for line-of-sight (LoS) links. Our joint DRL-based solution demonstrates an increase of up to 41% in the number of users served compared to scenarios with equal bandwidth and power allocation.
Paper Structure (27 sections, 6 equations, 9 figures, 2 tables, 1 algorithm)

This paper contains 27 sections, 6 equations, 9 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. System and Channel Models
  4. Problem Formulation
  5. Proposed Joint Model
  6. DQN Part Implementation
  7. Model Objective
  8. Initialization
  9. State and Observation Space
  10. Action Space
  11. State Updates
  12. Reward Function
  13. Penalty Function
  14. Training Termination
  15. Integration with DDPG Training
  16. ...and 12 more sections

Figures (9)

  • Figure 1: System model representing ground users in circular field served by a UAV
  • Figure 2: DQN training for bandwidth allocation with random power allocation and user positions
  • Figure 3: Joint DRL-based model's training with different user thresholds
  • Figure 4: Joint DRL-based model's training with different total bandwidth
  • Figure 5: Comparative analysis of joint DRL-based algorithm performance
  • ...and 4 more figures