Optimizing Path Planning using Deep Reinforcement Learning for UGVs in Precision Agriculture

TL;DR

This work investigates path planning for unmanned ground vehicles in precision agriculture using deep reinforcement learning across discrete and continuous action spaces. It compares traditional grid-based methods with DRL approaches, demonstrating that discrete DQN variants perform well on simple maps but struggle as complexity increases, while continuous-action TD3 consistently delivers robust, stable navigation in 2D and 3D simulations. A three-stage methodology progresses from 2D discrete to 2D continuous and finally to a Gazebo-based 3D environment, with transfer learning from static to dynamic obstacles enhancing performance. The study shows that continuous DRL, especially TD3, coupled with tailored reward shaping and ROS-Gazebo integration, yields high success rates and efficient paths, making a strong case for real-world deployment in precision agriculture for tasks such as targeted pesticide application. The results underscore the practicality of continuous DRL for robust, safe UGV operation amid moving obstacles in agricultural fields, offering a scalable path toward real-world agricultural robotics deployments.

Abstract

This study focuses on optimizing path planning for unmanned ground vehicles (UGVs) in precision agriculture using deep reinforcement learning (DRL) techniques in continuous action spaces. The research begins with a review of traditional grid-based methods, such as A* and Dijkstra's algorithms, and discusses their limitations in dynamic agricultural environments, highlighting the need for adaptive learning strategies. The study then explores DRL approaches, including Deep Q-Networks (DQN), which demonstrate improved adaptability and performance in two-dimensional simulations. Enhancements such as Double Q-Networks and Dueling Networks are evaluated to further improve decision-making. Building on these results, the focus shifts to continuous action space models, specifically Deep Deterministic Policy Gradient (DDPG) and Twin Delayed Deep Deterministic Policy Gradient (TD3), which are tested in increasingly complex environments. Experiments conducted in a three-dimensional environment using ROS and Gazebo demonstrate the effectiveness of continuous DRL algorithms in navigating dynamic agricultural scenarios. Notably, the pretrained TD3 agent achieves a 95 percent success rate in dynamic environments, demonstrating the robustness of the proposed approach in handling moving obstacles while ensuring safety for both crops and the robot.

Figures (18)

• Figure 1: Drones for Smart Agriculture - Module Diagram
• Figure 2: Methodology Diagram
• Figure 3: 2D Farm Environment in Pygame
• Figure 4: DQN Neural Network Architecture
• Figure 5: Performance of DQN in 8X8 Environment