Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Drone Stereo Vision for Radiata Pine Branch Detection and Distance Measurement: Utilizing Deep Learning and YOLO Integration

Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

TL;DR

This work presents an autonomous drone system for Radiata pine pruning that combines YOLO based branch segmentation with deep learning depth estimation to measure branch distance using stereo vision. It compares monocular and stereo depth methods, finding NeRF based stereo depth delivers the best accuracy albeit with slower inference, while SGBM remains faster but less complete. By integrating a triangulation based depth estimation with centroid and MAD based filtering, the approach achieves robust branch localization for pruning tasks on limited indoor data. The study highlights the potential for deep learning depth estimation to improve automated forestry operations, while acknowledging the need for larger, more diverse datasets to realize reliable field deployment.

Abstract

This research focuses on the development of a drone equipped with pruning tools and a stereo vision camera to accurately detect and measure the spatial positions of tree branches. YOLO is employed for branch segmentation, while two depth estimation approaches, monocular and stereo, are investigated. In comparison to SGBM, deep learning techniques produce more refined and accurate depth maps. In the absence of ground-truth data, a fine-tuning process using deep neural networks is applied to approximate optimal depth values. This methodology facilitates precise branch detection and distance measurement, addressing critical challenges in the automation of pruning operations. The results demonstrate notable advancements in both accuracy and efficiency, underscoring the potential of deep learning to drive innovation and enhance automation in the agricultural sector.

Drone Stereo Vision for Radiata Pine Branch Detection and Distance Measurement: Utilizing Deep Learning and YOLO Integration

TL;DR

This work presents an autonomous drone system for Radiata pine pruning that combines YOLO based branch segmentation with deep learning depth estimation to measure branch distance using stereo vision. It compares monocular and stereo depth methods, finding NeRF based stereo depth delivers the best accuracy albeit with slower inference, while SGBM remains faster but less complete. By integrating a triangulation based depth estimation with centroid and MAD based filtering, the approach achieves robust branch localization for pruning tasks on limited indoor data. The study highlights the potential for deep learning depth estimation to improve automated forestry operations, while acknowledging the need for larger, more diverse datasets to realize reliable field deployment.

Abstract

This research focuses on the development of a drone equipped with pruning tools and a stereo vision camera to accurately detect and measure the spatial positions of tree branches. YOLO is employed for branch segmentation, while two depth estimation approaches, monocular and stereo, are investigated. In comparison to SGBM, deep learning techniques produce more refined and accurate depth maps. In the absence of ground-truth data, a fine-tuning process using deep neural networks is applied to approximate optimal depth values. This methodology facilitates precise branch detection and distance measurement, addressing critical challenges in the automation of pruning operations. The results demonstrate notable advancements in both accuracy and efficiency, underscoring the potential of deep learning to drive innovation and enhance automation in the agricultural sector.
Paper Structure (10 sections, 2 equations, 9 figures)

This paper contains 10 sections, 2 equations, 9 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Branches Data Preparation and Application of the YOLO Instance Segmentation Method
  5. Deep learning Methods for Generating Depth Map
  6. Integration of Image Instance Segmentation and Depth Map Generation
  7. Results and Analysis
  8. Comparative research of Various Deep Learning Approaches for Depth Map Generation
  9. Final Results Achieved by Combining YOLO with NeRF Using the Second Combination Method
  10. Conclusions

Figures (9)

  • Figure 1: The drone, equipped with a ZED mini camera for stereo vision and a pruning tool autonomously detects and prunes branches of radiata pine. The ZED mini camera enables the drone to accurately identify the branches, while the pruning tool precisely prunes them.
  • Figure 2: Workflow Diagram of the Research Process
  • Figure 3: Prediction Results Using the YOLOv8n-seg Model Trained for 100 Epochs on the Branches Dataset
  • Figure 4: Comparison of Predicted Depth Maps Generated by MiDaS and Depth Anything Models at Branch Distances of 1m, 1.5m, and 2m from the Camera
  • Figure 5: Comparison of PSMNet Fine-tuning Results Across Different Pretrained Models and Training Epochs
  • ...and 4 more figures