Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EIA-SEC: Improved Actor-Critic Framework for Multi-UAV Collaborative Control in Smart Agriculture

Quanxi Zhou, Wencan Mao, Yilei Liang, Manabu Tsukada, Yunling Liu, Jon Crowcroft

TL;DR

The paper tackles coordinating a heterogeneous team of UAVs in smart agriculture under QoI constraints (AoI, VDF) and energy limits. It introduces EIA-SEC, a MARL framework that fuses Elite Imitation Actor guidance with a Shared Ensemble Critic to accelerate learning and curb Q-value overestimation. Empirical results show superior reward, faster convergence, and improved training stability versus strong MARL baselines, across scalability and ablation tests. This approach promises robust, real-time, cooperative UAV control for data collection, imaging, and communication in precision farming.

Abstract

The widespread application of wireless communication technology has promoted the development of smart agriculture, where unmanned aerial vehicles (UAVs) play a multifunctional role. We target a multi-UAV smart agriculture system where UAVs cooperatively perform data collection, image acquisition, and communication tasks. In this context, we model a Markov decision process to solve the multi-UAV trajectory planning problem. Moreover, we propose a novel Elite Imitation Actor-Shared Ensemble Critic (EIA-SEC) framework, where agents adaptively learn from the elite agent to reduce trial-and-error costs, and a shared ensemble critic collaborates with each agent's local critic to ensure unbiased objective value estimates and prevent overestimation. Experimental results demonstrate that EIA-SEC outperforms state-of-the-art baselines in terms of reward performance, training stability, and convergence speed.

EIA-SEC: Improved Actor-Critic Framework for Multi-UAV Collaborative Control in Smart Agriculture

TL;DR

The paper tackles coordinating a heterogeneous team of UAVs in smart agriculture under QoI constraints (AoI, VDF) and energy limits. It introduces EIA-SEC, a MARL framework that fuses Elite Imitation Actor guidance with a Shared Ensemble Critic to accelerate learning and curb Q-value overestimation. Empirical results show superior reward, faster convergence, and improved training stability versus strong MARL baselines, across scalability and ablation tests. This approach promises robust, real-time, cooperative UAV control for data collection, imaging, and communication in precision farming.

Abstract

The widespread application of wireless communication technology has promoted the development of smart agriculture, where unmanned aerial vehicles (UAVs) play a multifunctional role. We target a multi-UAV smart agriculture system where UAVs cooperatively perform data collection, image acquisition, and communication tasks. In this context, we model a Markov decision process to solve the multi-UAV trajectory planning problem. Moreover, we propose a novel Elite Imitation Actor-Shared Ensemble Critic (EIA-SEC) framework, where agents adaptively learn from the elite agent to reduce trial-and-error costs, and a shared ensemble critic collaborates with each agent's local critic to ensure unbiased objective value estimates and prevent overestimation. Experimental results demonstrate that EIA-SEC outperforms state-of-the-art baselines in terms of reward performance, training stability, and convergence speed.

Paper Structure

This paper contains 29 sections, 60 equations, 3 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. System Overview
  4. System Model
  5. UAV Communication Model
  6. Energy Consumption Model
  7. Data Collection Model
  8. Monitoring Model
  9. Design of EIA-SEC
  10. Markov Decision Process
  11. Agent
  12. Observation of Each Agent
  13. Action of Each Agent.
  14. Reward of Each Agent $r_{\text{c}-i,k}$, $r_{\text{m}-i,k}$, and $r_{\text{d}-i,k}$
  15. Elite Imitation Actor
  16. ...and 14 more sections

Figures (3)

  • Figure 1: The envisioned multi-UAV collaborative system for smart agriculture, where collection UAVs, monitoring UAVs, and communication UAVs cooperate with each other to minimize the system-level AoI and VDF.
  • Figure 2: The structure of our EIA-SEC framework.
  • Figure 3: Experimental results in the performance, scalability, ablation, and visualization studies.