Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trajectory Planning for UAV-Based Smart Farming Using Imitation-Based Triple Deep Q-Learning

Wencan Mao, Quanxi Zhou, Tomas Couso Coddou, Manabu Tsukada, Yunling Liu, Yusheng Ji

TL;DR

This work addresses UAV trajectory planning for smart farming under environmental uncertainty and partial observability by formulating the problem as an MDP and solving it with a novel imitation-based triple deep Q-network (ITDQN). ITDQN combines an elite imitation mechanism to reduce exploration costs with a mediator Q-network layered over a DDQN to enhance training speed, stability, and performance. Through synthetic simulations and real-world demonstrations, ITDQN consistently outperforms DQN, DDQN, and heuristic baselines, achieving higher weed recognition and data collection rates within UAV energy constraints. The approach enables scalable, multi-UAV coordination for weed management and sensor data collection, offering practical benefits for precision agriculture.

Abstract

Unmanned aerial vehicles (UAVs) have emerged as a promising auxiliary platform for smart agriculture, capable of simultaneously performing weed detection, recognition, and data collection from wireless sensors. However, trajectory planning for UAV-based smart agriculture is challenging due to the high uncertainty of the environment, partial observations, and limited battery capacity of UAVs. To address these issues, we formulate the trajectory planning problem as a Markov decision process (MDP) and leverage multi-agent reinforcement learning (MARL) to solve it. Furthermore, we propose a novel imitation-based triple deep Q-network (ITDQN) algorithm, which employs an elite imitation mechanism to reduce exploration costs and utilizes a mediator Q-network over a double deep Q-network (DDQN) to accelerate and stabilize training and improve performance. Experimental results in both simulated and real-world environments demonstrate the effectiveness of our solution. Moreover, our proposed ITDQN outperforms DDQN by 4.43\% in weed recognition rate and 6.94\% in data collection rate.

Trajectory Planning for UAV-Based Smart Farming Using Imitation-Based Triple Deep Q-Learning

TL;DR

This work addresses UAV trajectory planning for smart farming under environmental uncertainty and partial observability by formulating the problem as an MDP and solving it with a novel imitation-based triple deep Q-network (ITDQN). ITDQN combines an elite imitation mechanism to reduce exploration costs with a mediator Q-network layered over a DDQN to enhance training speed, stability, and performance. Through synthetic simulations and real-world demonstrations, ITDQN consistently outperforms DQN, DDQN, and heuristic baselines, achieving higher weed recognition and data collection rates within UAV energy constraints. The approach enables scalable, multi-UAV coordination for weed management and sensor data collection, offering practical benefits for precision agriculture.

Abstract

Unmanned aerial vehicles (UAVs) have emerged as a promising auxiliary platform for smart agriculture, capable of simultaneously performing weed detection, recognition, and data collection from wireless sensors. However, trajectory planning for UAV-based smart agriculture is challenging due to the high uncertainty of the environment, partial observations, and limited battery capacity of UAVs. To address these issues, we formulate the trajectory planning problem as a Markov decision process (MDP) and leverage multi-agent reinforcement learning (MARL) to solve it. Furthermore, we propose a novel imitation-based triple deep Q-network (ITDQN) algorithm, which employs an elite imitation mechanism to reduce exploration costs and utilizes a mediator Q-network over a double deep Q-network (DDQN) to accelerate and stabilize training and improve performance. Experimental results in both simulated and real-world environments demonstrate the effectiveness of our solution. Moreover, our proposed ITDQN outperforms DDQN by 4.43\% in weed recognition rate and 6.94\% in data collection rate.

Paper Structure

This paper contains 26 sections, 27 equations, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Trajectory Planning for UAV-Based Smart Farming
  4. Q-Learning, DQN, DDQN, and Variants
  5. UAV-Based Smart Farming
  6. Exemplary Scenario
  7. System Model
  8. Energy consumption model
  9. Communication model
  10. Markov Decision Process
  11. Agent
  12. Environment
  13. States $s_{i,t}$
  14. Actions $a_{i,t}$
  15. Reward $r_{i,t}$
  16. ...and 11 more sections

Figures (4)

  • Figure 1: Exemplary scenario of UAV-based smart farming, where UAVs perform weed detection, recognition, and data collection from wireless sensors simultaneously.
  • Figure 2: The overview of ITDQN.
  • Figure 3: Our evaluation follows a three-step methodology as follows. Step 1: Simulation in a synthetic field (a), step 2: Demonstration in an indoor area (b and c), and step 3: Evaluation in a real-world field as our future work (d).
  • Figure 4: Experimental Results.