Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Real-Time Branch-to-Tool Distance Estimation for Autonomous UAV Pruning: Benchmarking Five DEFOM-Stereo Variants from Simulation to Jetson Deployment

Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

Abstract

Autonomous tree pruning with unmanned aerial vehicles (UAVs) is a safety-critical real-world task: the onboard perception system must estimate the metric distance from a cutting tool to thin tree branches in real time so that the UAV can approach, align, and actuate the pruner without collision. We address this problem by training five variants of DEFOM-Stereo - a recent foundation-model-based stereo matcher - on a task-specific synthetic dataset and deploying the checkpoints on an NVIDIA Jetson Orin Super 16 GB. The training corpus is built in Unreal Engine 5 with a simulated ZED Mini stereo camera capturing 5,520 stereo pairs across 115 tree instances from three viewpoints at 2m distance; dense EXR depth maps provide exact, spatially complete supervision for thin branches. On the synthetic test set, DEFOM-Stereo ViT-S achieves the best depth-domain accuracy (EPE 1.74 px, D1-all 5.81%, delta-1 95.90%, depth MAE 23.40 cm) but its Jetson inference speed of ~2.2 FPS (~450 ms per frame) remains too slow for responsive closed-loop tool control. A newly introduced balanced variant, DEFOM-PrunePlus (~21M backbone, ~3.3 FPS on Jetson), offers the best deployable accuracy-speed trade-off (EPE 5.87 px, depth MAE 64.26 cm, delta-1 87.59%): its frame rate is sufficient for real-time guidance and its depth accuracy supports safe branch approach planning at the 2m operating range. The lightweight DEFOM-PruneStereo (~6.9 FPS) and DEFOM-PruneNano (~8.5 FPS) run fast but sacrifice substantial accuracy (depth MAE > 57 cm), making estimates too unreliable for safe actuation. Zero-shot inference on real photographs confirms that full-capacity models preserve branch geometry, validating the sim-to-real transfer. We conclude that DEFOM-PrunePlus provides the most practical accuracy-latency balance for onboard distance estimation, while ViT-S serves as the reference for future hardware.

Real-Time Branch-to-Tool Distance Estimation for Autonomous UAV Pruning: Benchmarking Five DEFOM-Stereo Variants from Simulation to Jetson Deployment

Abstract

Autonomous tree pruning with unmanned aerial vehicles (UAVs) is a safety-critical real-world task: the onboard perception system must estimate the metric distance from a cutting tool to thin tree branches in real time so that the UAV can approach, align, and actuate the pruner without collision. We address this problem by training five variants of DEFOM-Stereo - a recent foundation-model-based stereo matcher - on a task-specific synthetic dataset and deploying the checkpoints on an NVIDIA Jetson Orin Super 16 GB. The training corpus is built in Unreal Engine 5 with a simulated ZED Mini stereo camera capturing 5,520 stereo pairs across 115 tree instances from three viewpoints at 2m distance; dense EXR depth maps provide exact, spatially complete supervision for thin branches. On the synthetic test set, DEFOM-Stereo ViT-S achieves the best depth-domain accuracy (EPE 1.74 px, D1-all 5.81%, delta-1 95.90%, depth MAE 23.40 cm) but its Jetson inference speed of ~2.2 FPS (~450 ms per frame) remains too slow for responsive closed-loop tool control. A newly introduced balanced variant, DEFOM-PrunePlus (~21M backbone, ~3.3 FPS on Jetson), offers the best deployable accuracy-speed trade-off (EPE 5.87 px, depth MAE 64.26 cm, delta-1 87.59%): its frame rate is sufficient for real-time guidance and its depth accuracy supports safe branch approach planning at the 2m operating range. The lightweight DEFOM-PruneStereo (~6.9 FPS) and DEFOM-PruneNano (~8.5 FPS) run fast but sacrifice substantial accuracy (depth MAE > 57 cm), making estimates too unreliable for safe actuation. Zero-shot inference on real photographs confirms that full-capacity models preserve branch geometry, validating the sim-to-real transfer. We conclude that DEFOM-PrunePlus provides the most practical accuracy-latency balance for onboard distance estimation, while ViT-S serves as the reference for future hardware.

Paper Structure

This paper contains 32 sections, 2 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Stereo Matching for Dense Depth
  4. Efficient Stereo for Edge Deployment
  5. Foundation Models and Cross-Domain Depth
  6. Synthetic Data for Robotics and Forestry
  7. Task and Dataset
  8. Pruning-Oriented Distance Estimation
  9. UE5 Virtual Tree Branches Dataset
  10. Train, Validation, and Test Splits
  11. Real Tree-Branch Zero-Shot Set
  12. Models
  13. DEFOM-Stereo with ViT-S and ViT-L
  14. DEFOM-PrunePlus
  15. DEFOM-PruneStereo
  16. ...and 17 more sections

Figures (3)

  • Figure 1: Qualitative disparity comparison on four UE5 synthetic test scenes. Each row shows one scene; columns show the left RGB input, right RGB input, ground-truth disparity, and predicted disparity from the five models. ViT-S and ViT-L closely reproduce the GT structure, DEFOM-PrunePlus preserves scene layout with softer edges, while DEFOM-PruneStereo and DEFOM-PruneNano show increasing fragmentation and boundary blurring.
  • Figure 2: Onboard pruning pipeline on Jetson Orin Super. The stereo model provides the dense geometric signal that is converted into branch-to-tool distance for the cutting controller. At $\sim$3.3 FPS with DEFOM-PrunePlus, the system updates depth faster than the UAV can close the remaining distance.
  • Figure 3: Zero-shot transfer from UE5 training to real tree-branch photographs. Each row shows one real scene; columns show the left RGB input and predicted disparity from the five models. ViT-S and ViT-L preserve thin-branch continuity and boundary sharpness, DEFOM-PrunePlus retains reasonable structure but with softer boundaries, while DEFOM-PruneStereo and DEFOM-PruneNano show fragmented depth and blurred boundaries.