Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Drone Stereo Vision for Radiata Pine Branch Detection and Distance Measurement: Integrating SGBM and Segmentation Models

Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green

TL;DR

Experimental results demonstrate that the combined implementation of YOLO and SGBM enables the drone to accurately detect branches and measure their distances from the drone, a significant contribution to the advancement of drone technology in the automation of agricultural and forestry practices.

Abstract

Manual pruning of radiata pine trees presents significant safety risks due to their substantial height and the challenging terrains in which they thrive. To address these risks, this research proposes the development of a drone-based pruning system equipped with specialized pruning tools and a stereo vision camera, enabling precise detection and trimming of branches. Deep learning algorithms, including YOLO and Mask R-CNN, are employed to ensure accurate branch detection, while the Semi-Global Matching algorithm is integrated to provide reliable distance estimation. The synergy between these techniques facilitates the precise identification of branch locations and enables efficient, targeted pruning. Experimental results demonstrate that the combined implementation of YOLO and SGBM enables the drone to accurately detect branches and measure their distances from the drone. This research not only improves the safety and efficiency of pruning operations but also makes a significant contribution to the advancement of drone technology in the automation of agricultural and forestry practices, laying a foundational framework for further innovations in environmental management.

Drone Stereo Vision for Radiata Pine Branch Detection and Distance Measurement: Integrating SGBM and Segmentation Models

TL;DR

Experimental results demonstrate that the combined implementation of YOLO and SGBM enables the drone to accurately detect branches and measure their distances from the drone, a significant contribution to the advancement of drone technology in the automation of agricultural and forestry practices.

Abstract

Manual pruning of radiata pine trees presents significant safety risks due to their substantial height and the challenging terrains in which they thrive. To address these risks, this research proposes the development of a drone-based pruning system equipped with specialized pruning tools and a stereo vision camera, enabling precise detection and trimming of branches. Deep learning algorithms, including YOLO and Mask R-CNN, are employed to ensure accurate branch detection, while the Semi-Global Matching algorithm is integrated to provide reliable distance estimation. The synergy between these techniques facilitates the precise identification of branch locations and enables efficient, targeted pruning. Experimental results demonstrate that the combined implementation of YOLO and SGBM enables the drone to accurately detect branches and measure their distances from the drone. This research not only improves the safety and efficiency of pruning operations but also makes a significant contribution to the advancement of drone technology in the automation of agricultural and forestry practices, laying a foundational framework for further innovations in environmental management.
Paper Structure (14 sections, 12 equations, 6 figures, 1 table)

This paper contains 14 sections, 12 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Object detection and image segmentation
  4. Depth Map
  5. Performance Metrics
  6. Methods
  7. Data Collection and Image Instance Segmentation
  8. SGBM for Generating Depth Map
  9. Integration of Image Instance Segmentation and Depth Map Generation
  10. Results and Analysis
  11. Comparative Analysis of Mask R-CNN and YOLO Models for Object Detection and Image Segmentation
  12. SGBM depth map generation process
  13. Final Results Achieved by Combining YOLO with SGBM Using the First Combination Method
  14. Conclusions

Figures (6)

  • Figure 1: The drone, equipped with a ZED mini camera for stereo vision and a pruning tool autonomously detects and prunes branches of radiata pine. The ZED mini camera enables the drone to accurately identify the branches, while the pruning tool precisely prunes them.
  • Figure 2: Triangulation using Two Cameras to Obtain the Depth Map. The point $(u_l, v_l)$ represents the projection of point $p (x, y, z)$ in three-dimensional space onto the image plane of the left camera, whereas point $(u_r, v_r)$ corresponds to the projection of the same point onto the right camera’s image plane. The variable $b$ denotes the baseline distance separating the left and right cameras.$\hat{x}$, $\hat{y}$, and $\hat{z}$ represent the three axes of the camera or world coordinate frame, corresponding to the x, y, and z directions.
  • Figure 3: Research flow chart
  • Figure 4: (a) Present the results of predicted points spaced a certain distance apart, and (b) display the depth map obtained from SGBM. Combining these allows for determining the final distance between the branches and the stereo camera.
  • Figure 5: Show the original images, pre-processed images and disparity maps, (a) and (b) are the original left image and right image, after prepocessed we can get the (c) and (d), then we use SGBM to create the disparity map (e), then we through the WLS to get the (f).
  • ...and 1 more figures