Autonomous Apple Fruitlet Sizing with Next Best View Planning

TL;DR

This work tackles autonomous sizing of small apple fruitlets (diameters about $7$–$14$ mm) in cluttered orchards. Approach: a next-best-view planner that combines ROI-targeted sampling, attention-guided information gain, and a dual-map occupancy representation, followed by robust global registration, HCS clustering, and ellipse-based photogrammetric sizing. Contributions: a novel NBV planner, a robust data-association pipeline for multi-view fruitlet detections, and quantitative validation on both simulated and real orchard data. Findings: the planner yields higher sizing accuracy and fruitlet match rates than a state-of-the-art agriculture NBV planner, with strong linear correspondence to ground truth. Significance: enables autonomous, scalable sizing for thinning decisions and can be extended to other small fruits.

Abstract

In this paper, we present a next-best-view planning approach to autonomously size apple fruitlets. State-of-the-art viewpoint planners in agriculture are designed to size large and more sparsely populated fruit. They rely on lower resolution maps and sizing methods that do not generalize to smaller fruit sizes. To overcome these limitations, our method combines viewpoint sampling around semantically labeled regions of interest, along with an attention-guided information gain mechanism to more strategically select viewpoints that target the small fruits' volume. Additionally, we integrate a dual-map representation of the environment that is able to both speed up expensive ray casting operations and maintain the high occupancy resolution required to informatively plan around the fruit. When sizing, a robust estimation and graph clustering approach is introduced to associate fruit detections across images. Through simulated experiments, we demonstrate that our viewpoint planner improves sizing accuracy compared to state of the art and ablations. We also provide quantitative results on data collected by a real robotic system in the field.

Figures (11)

• Figure 1: Left: Stereo camera attached to a 7 DoF robotic arm. Right: Example of a fruitlet cluster with AprilTag hung in close proximity for identification.
• Figure 2: Overview of our next-best-view planning and sizing pipeline.
• Figure 3: (a) Original RGB image with AprilTag near cluster. (b) Extracted point cloud with fruitlet centroids. (c) Density map created by smoothing volume around centroids. (d) Sampling tree created from local maxima.
• Figure 4: Dual-map representation. Left: Coarse octree that stores occupancy information and spans the entire observation space. Right: Fine octree that stores occupancy and ROI information (green) within the Attention Region.
• Figure 5: Dual-map ray casting implementation. When ray casting, the coarse and fine maps are used outside and inside the Attention Region respectively.