Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lessons from Deploying CropFollow++: Under-Canopy Agricultural Navigation with Keypoints

Arun N. Sivakumar, Mateus V. Gasparino, Michael McGuire, Vitor A. H. Higuti, M. Ugur Akcal, Girish Chowdhary

TL;DR

This work tackles under-canopy autonomous navigation for crops with about 0.75 m row spacing using a vision-based approach that leverages semantic keypoints to infer heading and lateral position. CropFollow++ predicts heatmaps for three keypoints from RGB imagery and uses an MPC to generate velocity commands, with multi-camera fusion and a crash-recovery mode to enhance robustness. The authors validate the system through large-scale field deployments totaling 25 km on cover crop planting robots, analyze failure modes, and distill practical lessons for improving domain-shift resilience, calibration, and interpretability. The study demonstrates promising row-follow performance and highlights concrete directions—semi-supervised offline learning, online adaptation, and richer camera fusion—for achieving full-field under-canopy autonomy in agriculture.

Abstract

We present a vision-based navigation system for under-canopy agricultural robots using semantic keypoints. Autonomous under-canopy navigation is challenging due to the tight spacing between the crop rows ($\sim 0.75$ m), degradation in RTK-GPS accuracy due to multipath error, and noise in LiDAR measurements from the excessive clutter. Our system, CropFollow++, introduces modular and interpretable perception architecture with a learned semantic keypoint representation. We deployed CropFollow++ in multiple under-canopy cover crop planting robots on a large scale (25 km in total) in various field conditions and we discuss the key lessons learned from this.

Lessons from Deploying CropFollow++: Under-Canopy Agricultural Navigation with Keypoints

TL;DR

This work tackles under-canopy autonomous navigation for crops with about 0.75 m row spacing using a vision-based approach that leverages semantic keypoints to infer heading and lateral position. CropFollow++ predicts heatmaps for three keypoints from RGB imagery and uses an MPC to generate velocity commands, with multi-camera fusion and a crash-recovery mode to enhance robustness. The authors validate the system through large-scale field deployments totaling 25 km on cover crop planting robots, analyze failure modes, and distill practical lessons for improving domain-shift resilience, calibration, and interpretability. The study demonstrates promising row-follow performance and highlights concrete directions—semi-supervised offline learning, online adaptation, and richer camera fusion—for achieving full-field under-canopy autonomy in agriculture.

Abstract

We present a vision-based navigation system for under-canopy agricultural robots using semantic keypoints. Autonomous under-canopy navigation is challenging due to the tight spacing between the crop rows ( m), degradation in RTK-GPS accuracy due to multipath error, and noise in LiDAR measurements from the excessive clutter. Our system, CropFollow++, introduces modular and interpretable perception architecture with a learned semantic keypoint representation. We deployed CropFollow++ in multiple under-canopy cover crop planting robots on a large scale (25 km in total) in various field conditions and we discuss the key lessons learned from this.
Paper Structure (6 sections, 6 figures)

This paper contains 6 sections, 6 figures.

Table of Contents

  1. Introduction
  2. CropFollow++ Overview
  3. Field Experiments
  4. Field Experiment Results
  5. Analyzing Failure Modes
  6. Key Takeaways from CCR Deployment

Figures (6)

  • Figure 1: Cover Crop Robot navigating through the minimal space available between the crop rows.
  • Figure 2: CropFollow++ overview. The camera RGB image is used as input to our neural network model that predicts keypoints to locate the crop rows. The keypoints are used to create a trajectory that is used as the reference for an MPC controller.
  • Figure 3: We report the histogram of the distance traveled before intervention across all the autonomous runs from three CCRs. Though the majority of the runs are less than 250 m, we show three instances of autonomous runs with more than 2000 m.
  • Figure 4: Keypoint Detection in Action: Here we show three randomly sampled images from CCR deployment with keypoint detections. Left: Raw images from the CCR front camera. Center: Visualizations of the heatmaps for the vanishing point (red), the left point (green), and the right point (blue). Right: Visualizations of the keypoint outputs and the extracted vanishing lines, computed as the argmax of each heatmap.
  • Figure 5: We show the distribution of various causes of CCR autonomy interventions.
  • ...and 1 more figures