Learned Visual Navigation for Under-Canopy Agricultural Robots

TL;DR

The system, CropFollow, is able to autonomously drive 485 meters per intervention on average, outperforming a state-of-the-art LiDAR based system in extensive field testing spanning over 25 km.

Abstract

We describe a system for visually guided autonomous navigation of under-canopy farm robots. Low-cost under-canopy robots can drive between crop rows under the plant canopy and accomplish tasks that are infeasible for over-the-canopy drones or larger agricultural equipment. However, autonomously navigating them under the canopy presents a number of challenges: unreliable GPS and LiDAR, high cost of sensing, challenging farm terrain, clutter due to leaves and weeds, and large variability in appearance over the season and across crop types. We address these challenges by building a modular system that leverages machine learning for robust and generalizable perception from monocular RGB images from low-cost cameras, and model predictive control for accurate control in challenging terrain. Our system, CropFollow, is able to autonomously drive 485 meters per intervention on average, outperforming a state-of-the-art LiDAR based system (286 meters per intervention) in extensive field testing spanning over 25 km.

Figures (16)

• Figure 1: CropFollow is an autonomous navigation system for under-canopy agriculture robots. It uses RGB images from a front-facing camera to output steering commands to drive the robot in crop rows.
• Figure 2: CropFollow Overview. We use a convolutional network to output robot heading and placement in row. This is used to compute the row center which is used as a reference trajectory. A model predictive controller converts reference trajectories to angular velocity commands.
• Figure 3: Our method uses the robot's heading, $\phi$ and ratio of distance from the left and the right crop row, $d = d_L/(d_L+d_R)$, as the intermediate representation between perception and planning.
• Figure 4: Sample images from the collected dataset.
• Figure 5: Ground truthing procedure. Using the horizon annotations, we correct for the camera roll, and pitch. After this, heading, $\phi$ can be calculated by looking at the crop row vanishing point, and distance ratio can be computed from the intercepts of the crop row lines in the heading corrected image, as ${d_L}/({d_L+d_R})$.