A drone that learns to efficiently find objects in agricultural fields: from simulation to the real world

TL;DR

This work tackles efficient data collection in precision agriculture under drone battery constraints by training a reinforcement-learning path planner in an abstract simulation with uncertain priors and a detection network. The RL agent, based on a Deep Q-Network, is deployed onboard a real drone by mapping real observations to a drone-centered state and translating actions to coordinates, aided by a prior-knowledge map generated from a high-altitude survey. Across four realism levels, the agent achieves substantially shorter flight paths than a full-coverage baseline (roughly 72–78% shorter) but at the cost of reduced recall (approximately 14–25% lower), with real-world performance limited by object distribution and prior-knowledge accuracy. The results suggest the approach is practical for applications prioritizing speed over completeness (e.g., weed detection) and highlight avenues for improvement through more robust priors, better detectors, and adaptive altitude strategies.

Abstract

Drones are promising for data collection in precision agriculture, however, they are limited by their battery capacity. Efficient path planners are therefore required. This paper presents a drone path planner trained using Reinforcement Learning (RL) on an abstract simulation that uses object detections and uncertain prior knowledge. The RL agent controls the flight direction and can terminate the flight. By using the agent in combination with the drone's flight controller and a detection network to process camera images, it is possible to evaluate the performance of the agent on real-world data. In simulation, the agent yielded on average a 78% shorter flight path compared to a full coverage planner, at the cost of a 14% lower recall. On real-world data, the agent showed a 72% shorter flight path compared to a full coverage planner, however, at the cost of a 25% lower recall. The lower performance on real-world data was attributed to the real-world object distribution and the lower accuracy of prior knowledge, and shows potential for improvement. Overall, we concluded that for applications where it is not crucial to find all objects, such as weed detection, the learned-based path planner is suitable and efficient.

Figures (6)

• Figure 1: Overview of the learning-based adaptive path planner, which learns a policy in simulation that can be applied in the real world. The agent is trained in an abstract simulation using a Deep Q-Network to select the best action $a_t$ given state $s_t$ that maximizes reward $r_t$ in the long term. During application, the drone's flight controller executes the action $a_t$ and the output from the detection network is converted to state $s_{t+1}$.
• Figure 2: DQN architecture, showing the pooling, convolutional, and fully connected layers with the input size, the number of kernels and their size, and the size of the flatten layer. Modified from vanEssen2024.
• Figure 3: Example of the artificial plants in the grass field.
• Figure 4: Mean reward of the evaluation simulations during training of the DQN agent. The line shows the running average over 1M timesteps.
• Figure 5: Effect of flight path length on the recall for the RL agent (DQN) and the baseline full coverage planner (FCov) with the different realism levels. Note that the lines for 'FCov level 2' and 'FCov level 3' are equal.