Sparse Data Tree Canopy Segmentation: Fine-Tuning Leading Pretrained Models on Only 150 Images

TL;DR

The paper tackles canopy mapping from aerial imagery under severe data scarcity by comparing five architectures—YOLOv11 Seg, Mask R-CNN, DeepLabV3, Swin-UNet, and DINOv2—fine-tuned on a 150-image dataset. It finds convolution-based models outperform transformer-based ones in terms of instance-level generalization, with YOLOv11 and Mask R-CNN achieving higher weighted mAP, while transformer approaches struggle without extensive pretraining or augmentation. The study highlights the importance of inductive biases and appropriate task framing (instance vs semantic segmentation) when data are scarce. Practically, the results advocate using lightweight CNNs with strong pretraining for canopy segmentation in low-data remote sensing contexts, informing applications in carbon accounting and wildfire monitoring.

Abstract

Tree canopy detection from aerial imagery is an important task for environmental monitoring, urban planning, and ecosystem analysis. Simulating real-life data annotation scarcity, the Solafune Tree Canopy Detection competition provides a small and imbalanced dataset of only 150 annotated images, posing significant challenges for training deep models without severe overfitting. In this work, we evaluate five representative architectures, YOLOv11, Mask R-CNN, DeepLabv3, Swin-UNet, and DINOv2, to assess their suitability for canopy segmentation under extreme data scarcity. Our experiments show that pretrained convolution-based models, particularly YOLOv11 and Mask R-CNN, generalize significantly better than pretrained transformer-based models. DeeplabV3, Swin-UNet and DINOv2 underperform likely due to differences between semantic and instance segmentation tasks, the high data requirements of Vision Transformers, and the lack of strong inductive biases. These findings confirm that transformer-based architectures struggle in low-data regimes without substantial pretraining or augmentation and that differences between semantic and instance segmentation further affect model performance. We provide a detailed analysis of training strategies, augmentation policies, and model behavior under the small-data constraint and demonstrate that lightweight CNN-based methods remain the most reliable for canopy detection on limited imagery.

Figures (2)

• Figure 1: Examples of Training Images with Different Resolutions and Scenes and their Segmentation Labels.
• Figure 2: Qualitative Tree canopy Segmentation Results. Left to Right: Raw image, YOLOv11, Mask-RCNN, DeepLabv3, Swin-UNet, DinoV2