Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cooperative UAVs for Remote Data Collection under Limited Communications: An Asynchronous Multiagent Learning Framework

Cuong Le, Symeon Chatzinotas, Thang X. Vu

TL;DR

This work tackles cooperative data collection by multiple UAVs under asynchronous decision-making and limited inter-UAV communication. It formulates the trajectory optimization as a Dec-POSMDP and introduces Asynchronous-QMIX (AQMIX) to learn cooperative policies, with state-downsampling to improve scalability. After learning, bandwidth allocation under imperfect CSI is solved via convex optimization, enhancing hovering efficiency. The results show that the proposed framework achieves higher energy efficiency, shorter mission times, and robust performance across varying UAV counts, data densities, and channel conditions, highlighting its practical potential for remote sensing and data-gathering missions.

Abstract

This paper addresses the joint optimization of trajectories and bandwidth allocation for multiple Unmanned Aerial Vehicles (UAVs) to enhance energy efficiency in the cooperative data collection problem. We focus on an important yet underestimated aspect of the system, where action synchronization across all UAVs is impossible. Since most existing learning-based solutions are not designed to learn in this asynchronous environment, we formulate the trajectory planning problem as a Decentralized Partially Observable Semi-Markov Decision Process and introduce an asynchronous multi-agent learning algorithm to learn UAVs' cooperative policies. Once the UAVs' trajectory policies are learned, the bandwidth allocation can be optimally solved based on local observations at each collection point. Comprehensive empirical results demonstrate the superiority of the proposed method over other learning-based and heuristic baselines in terms of both energy efficiency and mission completion time. Additionally, the learned policies exhibit robustness under varying environmental conditions.

Cooperative UAVs for Remote Data Collection under Limited Communications: An Asynchronous Multiagent Learning Framework

TL;DR

This work tackles cooperative data collection by multiple UAVs under asynchronous decision-making and limited inter-UAV communication. It formulates the trajectory optimization as a Dec-POSMDP and introduces Asynchronous-QMIX (AQMIX) to learn cooperative policies, with state-downsampling to improve scalability. After learning, bandwidth allocation under imperfect CSI is solved via convex optimization, enhancing hovering efficiency. The results show that the proposed framework achieves higher energy efficiency, shorter mission times, and robust performance across varying UAV counts, data densities, and channel conditions, highlighting its practical potential for remote sensing and data-gathering missions.

Abstract

This paper addresses the joint optimization of trajectories and bandwidth allocation for multiple Unmanned Aerial Vehicles (UAVs) to enhance energy efficiency in the cooperative data collection problem. We focus on an important yet underestimated aspect of the system, where action synchronization across all UAVs is impossible. Since most existing learning-based solutions are not designed to learn in this asynchronous environment, we formulate the trajectory planning problem as a Decentralized Partially Observable Semi-Markov Decision Process and introduce an asynchronous multi-agent learning algorithm to learn UAVs' cooperative policies. Once the UAVs' trajectory policies are learned, the bandwidth allocation can be optimally solved based on local observations at each collection point. Comprehensive empirical results demonstrate the superiority of the proposed method over other learning-based and heuristic baselines in terms of both energy efficiency and mission completion time. Additionally, the learned policies exhibit robustness under varying environmental conditions.
Paper Structure (30 sections, 1 theorem, 35 equations, 13 figures, 1 algorithm)

This paper contains 30 sections, 1 theorem, 35 equations, 13 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. System Model and Problem Formulation
  4. UAVs' Trajectories and Energy Consumption
  5. Data Transmission Model
  6. Problem Formulation
  7. Trajectory Optimization under Dec-POSMDP
  8. Dec-POSMDP Framework for Trajectory Optimization
  9. Detailed Components
  10. UAVs' termination conditions
  11. State space
  12. Observation
  13. Action space
  14. Communication message between UAVs
  15. Reward function
  16. ...and 15 more sections

Key Result

Lemma 1

Given that $\frac{\partial Q_{\texttt{tot}}}{\partial Q^n} \geq 0\ \forall n \in \mathcal{N}$, and that then we have

Figures (13)

  • Figure 1: Illustration of the investigated system. Blue links represent inter-UAV communication, and orange links represent SN-UAV data transmission. Cells with dark green and bold borders under UAVs represent their observable regions.
  • Figure 2: The architecture of the proposed algorithm. The green, blue, and red blocks represent UAVs' policy networks, mixing network, and the hypernetwork. A downsampling layer is included in the hypernetwork to reduce the state space.
  • Figure 3: Distribution of SNs over the 8$\times$8 cell collecting area. The colors of SNs represent an example of data collection demands with available data size ranging from 0 to 1 Mbits.
  • Figure 4: Total reward per episode during training on different network sizes.
  • Figure 5: Total reward per episode during training with different numbers of UAVs.
  • ...and 8 more figures

Theorems & Definitions (2)

  • Lemma 1
  • proof