Bridging Classical and Modern Computer Vision: PerceptiveNet for Tree Crown Semantic Segmentation

TL;DR

This work tackles tree crown semantic segmentation in remote sensing by introducing PerceptiveNet, a CNN-based architecture with a trainable Log-Gabor convolutional layer that enhances spatial-frequency feature extraction and a backbone engineered for a wider receptive field. The authors also build a hybrid CNN-Transformer variant (PerceptiveNeTr) and compare against strong CNN and Transformer baselines across TreeCrown, Landcover.AI, and UAVid, demonstrating state-of-the-art performance and strong cross-domain generalisation. Ablation studies confirm that Log-Gabor, mixed pooling, and dilated convolutions offer complementary gains, and CAM-based analyses show more focused feature representations. The results indicate practical impact for forest management, biodiversity monitoring, and carbon accounting, showcasing robust segmentation even under shadows, occlusions, and diverse canopy structures.

Abstract

The accurate semantic segmentation of tree crowns within remotely sensed data is crucial for scientific endeavours such as forest management, biodiversity studies, and carbon sequestration quantification. However, precise segmentation remains challenging due to complexities in the forest canopy, including shadows, intricate backgrounds, scale variations, and subtle spectral differences among tree species. Compared to the traditional methods, Deep Learning models improve accuracy by extracting informative and discriminative features, but often fall short in capturing the aforementioned complexities. To address these challenges, we propose PerceptiveNet, a novel model incorporating a Logarithmic Gabor-parameterised convolutional layer with trainable filter parameters, alongside a backbone that extracts salient features while capturing extensive context and spatial information through a wider receptive field. We investigate the impact of Log-Gabor, Gabor, and standard convolutional layers on semantic segmentation performance through extensive experimentation. Additionally, we conduct an ablation study to assess the contributions of individual layers and their combinations to overall model performance, and we evaluate PerceptiveNet as a backbone within a novel hybrid CNN-Transformer model. Our results outperform state-of-the-art models, demonstrating significant performance improvements on a tree crown dataset while generalising across domains, including two benchmark aerial scene semantic segmentation datasets with varying complexities.

Figures (6)

• Figure 1: Tree Crown Semantic Segmentation, depicting (from left to right): Original Image; Corresponding Mask; ResUNet; and proposed model's semantic segmentation. Each colour represents a different tree species. The images portray densely packed trees with complex boundaries due to partial overlap. Moreover, tree crown similarities, complex occlusion, combined with light variations and shadows further augment the scene's complexity.
• Figure 2: Dense forest canopy, demonstrating the impact of shadows, light variations, overlapping tree crowns, and weak distinctive features among tree species on the tree crown segmentation.
• Figure 3: Building blocks of the proposed Architecture: (a) PerceptiveNet architecture, (b) Decoder proposed dilated residual unit (DilRes), (c) Encoder/Bridge proposed dilated residual unit (DilRes), comprised of a mixture of average and maximum pooling layer (Mix Pool) and an averaged Dilated convolutional layer, (d) Dilated convolutional layer. Noticeably, the Mix Pool layer is not present on the decoder.
• Figure 4: PerceptiveNeTr: Hybrid CNN (PerceptiveNet) - Transformer architecture, leveraging long-range dependencies and global context.
• Figure 5: Class Activation Maps, ResUNet vs PerceptiveNet Encoder feature extraction capabilities. PerceptiveNet: More focused and detailed activation pattern.