Towards Safe and Efficient Through-the-Canopy Autonomous Fruit Counting with UAVs

TL;DR

The paper tackles the problem of accurate fruit counting in large orchards from canopy-level UAV flights, where occlusions and varied canopy structure complicate data collection. It presents a three-part approach: (1) a high-fidelity orchard simulation using Helios 3D to optimize global trajectories with occlusion-aware visibility analysis; (2) a low-cost autonomous canopy-navigation stack (RAPPIDS for obstacle avoidance and OpenVINS for VIO-based state estimation) implemented on a lightweight hardware platform; and (3) an RGB-based fruit counting pipeline combining YOLOv8 detection, ByteTrack tracking, COLMAP 3D reconstruction, and DBSCAN clustering to mitigate double counting. The framework demonstrates improved visibility via canopy-level data collection and provides an end-to-end workflow from planning to counting, validated through experimental flights and counting demonstrations. Together, these contributions enable scalable, efficient yield estimation in dense orchards, with potential impact on harvest scheduling and resource allocation.

Abstract

We present an autonomous aerial system for safe and efficient through-the-canopy fruit counting. Aerial robot applications in large-scale orchards face significant challenges due to the complexity of fine-tuning flight paths based on orchard layouts, canopy density, and plant variability. Through-the-canopy navigation is crucial for minimizing occlusion by leaves and branches but is more challenging due to the complex and dense environment compared to traditional over-the-canopy flights. Our system addresses these challenges by integrating: i) a high-fidelity simulation framework for optimizing flight trajectories, ii) a low-cost autonomy stack for canopy-level navigation and data collection, and iii) a robust workflow for fruit detection and counting using RGB images. We validate our approach through fruit counting with canopy-level aerial images and by demonstrating the autonomous navigation capabilities of our experimental vehicle.

Figures (8)

• Figure 1: Experimental vehicle flying autonomously at canopy level while collecting visual data using a RGB camera. This figure is a composite image showing multiple states of the vehicle during flight.
• Figure 2: Block diagram of the proposed autonomous system, consisting of three main components: a simulation framework that leverages specified orchard parameters (such as average tree height, tree spacing, and plant type) for global planning; onboard autonomy components that ensure safe and efficient data collection; and a post-flight counting algorithm that processes the collected RGB images, producing the final fruit count.
• Figure 3: Illustration of the occlusion checking process. Frustum culling is first performed, where only the geometric primitives contained in the camera's field of view are considered. A ray is cast from the camera to each contained fruit's center, and a fruit is marked as visible only if the ray does not intersect any surrounding mesh triangles.
• Figure 4: Simulation results comparing fruit counts using three distinct data collection strategies. (a)-(c) depict camera configurations for through-the-canopy flight, over-canopy flight, and ground vehicle collection, respectively. (d) shows the lawn-mower pattern applied uniformly across all methods for comparison. (e)-(g) present fruit counting results and the corresponding fruit distribution patterns. Data collection using the lawn-mower pattern in the simulation captured 66%, 55%, and 34% of the total fruits for each method, respectively, highlighting the advantage of through-the-canopy data collection using UAVs.
• Figure 5: Simulation results comparing fruit counting performance at different through-the-canopy flight heights and camera configurations. (a) Simulated trajectories tested at heights ranging from 1 m to 8 m, using two configurations: a single front-facing camera and two side-facing cameras. (b) Results showing fruit visibility at different flight heights, with optimal heights of 6.5 m for the front-facing camera and 5 m for the side-facing cameras. The two side-facing cameras achieve a maximum visible fruit coverage of 45.4%, compared to 32.3% for the front-facing camera.