Active Robotic Perception for Disease Detection and Mapping in Apple Trees

Abstract

Large-scale orchard production requires timely and precise disease monitoring, yet routine manual scouting is labor-intensive and financially impractical at the scale of modern operations. As a result, disease outbreaks are often detected late and tracked at coarse spatial resolutions, typically at the orchard-block level. We present an autonomous mobile active perception system for targeted disease detection and mapping in dormant apple trees, demonstrated on one of the most devastating diseases affecting apple today -- fire blight. The system integrates flash-illuminated stereo RGB sensing, real-time depth estimation, instance-level segmentation, and confidence-aware semantic 3D mapping to achieve precise localization of disease symptoms. Semantic predictions are fused into the volumetric occupancy map representation enabling the tracking of both occupancy and per-voxel semantic confidence, building actionable spatial maps for growers. To actively refine observations within complex canopies, we evaluate three viewpoint planning strategies within a unified perception-action loop: a deterministic geometric baseline, a volumetric next-best-view planner that maximizes unknown-space reduction, and a semantic next-best-view planner that prioritizes low-confidence symptomatic regions. Experiments on a fabricated lab tree and five simulated symptomatic trees demonstrate reliable symptom localization and mapping as a precursor to a field evaluation. In simulation, the semantic planner achieves the highest F1 score (0.6106) after 30 viewpoints, while the volumetric planner achieves the highest ROI coverage (85.82\%). In the lab setting, the semantic planner attains the highest final F1 (0.9058), with both next-best-view planners substantially improving coverage over the baseline.

Figures (6)

• Figure 1: Autonomous disease inspection system (left) and representative fire blight symptoms (right). Shoot blight instances exhibit a characteristic shepherd's crook (top), while canker regions appear as sunken or darkened lesions on woody tissue (bottom).
• Figure 2: Effect of our flash camera on a canker in the afternoon. Image on the left was captured using an iPhone 15 max and the image on the right was captured using the flash camera detailed in Silwal2021FlashCamera which is the primary camera used in our robotic system.
• Figure 3: The perception pipeline (top) generates semantic 3D reconstructions from stereo depth and instance segmentation. The viewpoint planning module (bottom) first generates candidate viewpoints using a plane-based sampling strategy (baseline planner), augments them with orientation variations, and then selects viewpoints by clustering either frontier voxels (volumetric planner) or semantically uncertain regions (semantic planner) and sampling poses oriented toward the cluster centroids.
• Figure 4: Blender-rendered symptomatic apple trees used in simulated Gazebo evaluations. Each tree contains randomly distributed shepherd’s crooks and cankers to simulate realistic outbreak variability and geometric complexity.
• Figure 5: Laboratory setup with the Husky mobile base, xArm6 manipulator, and flash stereo camera inspecting a fabricated symptomatic tree. The black backdrop shown is purely for cosmetic purposes and is not required for system operation.