Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Learning Meets Dynamics: Distributed User Connectivity Maximization in UAV-Based Communication Networks

Bowei Li, Saugat Tripathi, Salman Hosain, Ran Zhang, Jiang, Xie, Miao Wang

TL;DR

This work addresses maximizing user connectivity in UAV-based communication networks under dynamic UAV crews and shifting user distributions by formulating a time-coupled, mixed-integer non-convex problem and solving it with distributed MARL. It introduces two algorithms, DUCM-1 and DUCM-2, that explore four information-exchange levels and heterogeneous training to balance convergence, performance, and adaptation. Results show that exchanging problem-specific information (Level 3) yields the best convergence, and DUCM-2 robustly handles arbitrary quits and joins with a single training. The proposed framework offers a scalable, distributed approach for adaptive UAV trajectory control in realistic, dynamic UCNs with potential real-world impact for on-demand connectivity provisioning.

Abstract

Distributed management over Unmanned Aerial Vehicle (UAV) based communication networks (UCNs) has attracted increasing research attention. In this work, we study a distributed user connectivity maximization problem in a UCN. The work features a horizontal study over different levels of information exchange during the distributed iteration and a consideration of dynamics in UAV set and user distribution, which are not well addressed in the existing works. Specifically, the studied problem is first formulated into a time-coupled mixed-integer non-convex optimization problem. A heuristic two-stage UAV-user association policy is proposed to faster determine the user connectivity. To tackle the NP-hard problem in scalable manner, the distributed user connectivity maximization algorithm 1 (DUCM-1) is proposed under the multi-agent deep Q learning (MA-DQL) framework. DUCM-1 emphasizes on designing different information exchange levels and evaluating how they impact the learning convergence with stationary and dynamic user distribution. To comply with the UAV dynamics, DUCM-2 algorithm is developed which is devoted to autonomously handling arbitrary quit's and join-in's of UAVs in a considered time horizon. Extensive simulations are conducted i) to conclude that exchanging state information with a deliberated task-specific reward function design yields the best convergence performance, and ii) to show the efficacy and robustness of DUCM-2 against the dynamics.

When Learning Meets Dynamics: Distributed User Connectivity Maximization in UAV-Based Communication Networks

TL;DR

This work addresses maximizing user connectivity in UAV-based communication networks under dynamic UAV crews and shifting user distributions by formulating a time-coupled, mixed-integer non-convex problem and solving it with distributed MARL. It introduces two algorithms, DUCM-1 and DUCM-2, that explore four information-exchange levels and heterogeneous training to balance convergence, performance, and adaptation. Results show that exchanging problem-specific information (Level 3) yields the best convergence, and DUCM-2 robustly handles arbitrary quits and joins with a single training. The proposed framework offers a scalable, distributed approach for adaptive UAV trajectory control in realistic, dynamic UCNs with potential real-world impact for on-demand connectivity provisioning.

Abstract

Distributed management over Unmanned Aerial Vehicle (UAV) based communication networks (UCNs) has attracted increasing research attention. In this work, we study a distributed user connectivity maximization problem in a UCN. The work features a horizontal study over different levels of information exchange during the distributed iteration and a consideration of dynamics in UAV set and user distribution, which are not well addressed in the existing works. Specifically, the studied problem is first formulated into a time-coupled mixed-integer non-convex optimization problem. A heuristic two-stage UAV-user association policy is proposed to faster determine the user connectivity. To tackle the NP-hard problem in scalable manner, the distributed user connectivity maximization algorithm 1 (DUCM-1) is proposed under the multi-agent deep Q learning (MA-DQL) framework. DUCM-1 emphasizes on designing different information exchange levels and evaluating how they impact the learning convergence with stationary and dynamic user distribution. To comply with the UAV dynamics, DUCM-2 algorithm is developed which is devoted to autonomously handling arbitrary quit's and join-in's of UAVs in a considered time horizon. Extensive simulations are conducted i) to conclude that exchanging state information with a deliberated task-specific reward function design yields the best convergence performance, and ii) to show the efficacy and robustness of DUCM-2 against the dynamics.
Paper Structure (27 sections, 19 equations, 12 figures, 1 table, 2 algorithms)

This paper contains 27 sections, 19 equations, 12 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. System Model
  4. Network Model
  5. Spectrum Access
  6. User Admission and Association
  7. Problem Formulation
  8. Design of Distributed User Connectivity Maximization Algorithm 1 (DUCM-1 Algorithm)
  9. State Space
  10. Action Space
  11. Design of Information Exchange and Reward Function
  12. Implicit Information Exchange (Level 1)
  13. Exchange of Individual Reward Information (Level 2)
  14. Exchange of Problem-Specific Information (Level 3)
  15. Exchange of Complete Local State Information (Level 4)
  16. ...and 12 more sections

Figures (12)

  • Figure 1: Network Model.
  • Figure 2: Convergence of the number of connected users per episode for different levels of information exchange.
  • Figure 3: Final UAV positions and user connectivity under Level 3 and Level 4
  • Figure 4: Convergence comparison between MA-QL and MA-DQL.
  • Figure 5: Convergence of different levels with dynamic user distribution.
  • ...and 7 more figures