On-the-Go Tree Detection and Geometric Traits Estimation with Ground Mobile Robots in Fruit Tree Groves

TL;DR

This paper addresses real-time, on-the-go detection of trees and estimation of geometric traits in fruit-tree groves using a ground mobile robot. It fuses 2D NDVI information from an RG N camera with 3D LiDAR data, and employs a multivariate, entropy-based landmark association integrated into a Kalman-filter framework to localize trees against a georeferenced map while simultaneously estimating tree width and height. The approach avoids relying on specific tree parts, demonstrates robust performance in simulation and real-field tests, and proves capable of operating with onboard sensing and computation even under challenging wind and lighting conditions. This work advances precision agriculture by enabling autonomous, proximal sensing for per-tree management decisions and sets the stage for multi-robot expansion and richer health/yield indicators derived from multi-modal data.

Abstract

By-tree information gathering is an essential task in precision agriculture achieved by ground mobile sensors, but it can be time- and labor-intensive. In this paper we present an algorithmic framework to perform real-time and on-the-go detection of trees and key geometric characteristics (namely, width and height) with wheeled mobile robots in the field. Our method is based on the fusion of 2D domain-specific data (normalized difference vegetation index [NDVI] acquired via a red-green-near-infrared [RGN] camera) and 3D LiDAR point clouds, via a customized tree landmark association and parameter estimation algorithm. The proposed system features a multi-modal and entropy-based landmark correspondences approach, integrated into an underlying Kalman filter system to recognize the surrounding trees and jointly estimate their spatial and vegetation-based characteristics. Realistic simulated tests are used to evaluate our proposed algorithm's behavior in a variety of settings. Physical experiments in agricultural fields help validate our method's efficacy in acquiring accurate by-tree information on-the-go and in real-time by employing only onboard computational and sensing resources.

Figures (6)

• Figure 1: (a) Clearpath Jackal equipped with a Velodyne VLP-16 LiDAR and a MAPIR Survey3N RGN camera during field experiments. (b) The designed simulation world in Gazebo along with the simulated Jackal mobile robot. This world model contains a $2\times 3$ tree grid of 3 orange and 3 lemon trees placed at a $5~m$ width $\times$$5~m$ length relative distance from each other.
• Figure 2: Snapshot from robot operation in the orchard. (a) RGN raw imagery data. (b) Corresponding binary thresholded NDVI frame with projected 3D LiDAR data colorized based on $z$-level value.
• Figure 3: System diagram of the proposed 2D-3D fusion system with main sub-components, along with the positioning node, the field map node, and the raw sensory input streams. The solid lines indicate the inner connectivity of the components and the dotted arrows the information exchange between the developed nodes.
• Figure 4: (a) In height estimation, the robot senses a change on top $ring\_id$ detection of the cluster $\mathcal{C}_k$ and thus registers that height value with respect to $\{base\_footprint\}$. (b) In width estimation, the robot computes a tree's width value for each of the 10 $z$-slices of the clusters and stores the maximum one.
• Figure 5: Panoramic views of the simulated tests for (a) straight-line, (b) $N$-type, and (c) $S$-type robot trajectories on the $5~m$ width $\times$$5~m$ length tree grid setup. Along with the sampled and decreasingly transparent (with respect to time) robot poses, there is a colorized illustration of the detected tree candidates' $p_{centroid}$ points, which are projected on the UTM $x-y$ plane to demonstrate their distribution around the groundtruth $\mathcal{GT}$ tree positions.