Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Image-based Tree Crown Segmentation from Enhanced Lidar-based Pseudo-labels

Julius Pesonen, Stefan Rua, Josef Taher, Niko Koivumäki, Xiaowei Yu, Eija Honkavaara

TL;DR

This study presents a method to train deep learning models that segment and separate individual trees from RGB and multispectral images, using pseudo-labels derived from aerial laser scanning (ALS) data, and shows that the ALS-derived pseudo-labels can be enhanced using a zero-shot instance segmentation model, Segment Anything Model 2 (SAM 2).

Abstract

Mapping individual tree crowns is essential for tasks such as maintaining urban tree inventories and monitoring forest health, which help us understand and care for our environment. However, automatically separating the crowns from each other in aerial imagery is challenging due to factors such as the texture and partial tree crown overlaps. In this study, we present a method to train deep learning models that segment and separate individual trees from RGB and multispectral images, using pseudo-labels derived from aerial laser scanning (ALS) data. Our study shows that the ALS-derived pseudo-labels can be enhanced using a zero-shot instance segmentation model, Segment Anything Model 2 (SAM 2). Our method offers a way to obtain domain-specific training annotations for optical image-based models without any manual annotation cost, leading to segmentation models which outperform any available models which have been targeted for general domain deployment on the same task.

Learning Image-based Tree Crown Segmentation from Enhanced Lidar-based Pseudo-labels

TL;DR

This study presents a method to train deep learning models that segment and separate individual trees from RGB and multispectral images, using pseudo-labels derived from aerial laser scanning (ALS) data, and shows that the ALS-derived pseudo-labels can be enhanced using a zero-shot instance segmentation model, Segment Anything Model 2 (SAM 2).

Abstract

Mapping individual tree crowns is essential for tasks such as maintaining urban tree inventories and monitoring forest health, which help us understand and care for our environment. However, automatically separating the crowns from each other in aerial imagery is challenging due to factors such as the texture and partial tree crown overlaps. In this study, we present a method to train deep learning models that segment and separate individual trees from RGB and multispectral images, using pseudo-labels derived from aerial laser scanning (ALS) data. Our study shows that the ALS-derived pseudo-labels can be enhanced using a zero-shot instance segmentation model, Segment Anything Model 2 (SAM 2). Our method offers a way to obtain domain-specific training annotations for optical image-based models without any manual annotation cost, leading to segmentation models which outperform any available models which have been targeted for general domain deployment on the same task.
Paper Structure (21 sections, 1 equation, 9 figures, 4 tables)

This paper contains 21 sections, 1 equation, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Materials and methods
  4. Data
  5. Coarse segments
  6. SAM 2 enhancement
  7. Pseudo-supervised model
  8. Evaluation
  9. Baselines
  10. Detectree2
  11. Supervised U-net
  12. Grounded SAM (2)
  13. DeepForest + SAM 2
  14. SAM 3
  15. Experiments and results
  16. ...and 6 more sections

Figures (9)

  • Figure 1: RGB visualisation of the orthophoto spanning the whole dataset region on the left, with the manually annotated test region delineated in light green and shown separately on the right. The manual test annotations are delineated in white.
  • Figure 2: Visualisation of the different labels on the test set.
  • Figure 3: Overview of the proposed data fusion method for the pseudo-supervised model training.
  • Figure 4: Visualisation of the outputs of selected models. The Detectree2 results are from the best performing Flexi checkpoint.
  • Figure 5: Standard quartile box plot showing the prediction quality (mIoU) between different input modalities and coarse vs. pseudo-supervision.
  • ...and 4 more figures