AdaTreeFormer: Few Shot Domain Adaptation for Tree Counting from a Single High-Resolution Image

TL;DR

This work tackles tree counting from a single high-resolution image under strong cross-domain shifts. It introduces AdaTreeFormer, a transformer-based framework with a shared encoder and three domain-adaptive decoders that employ an attention-to-adapt mechanism and hierarchical cross-domain feature alignment, augmented by adversarial learning, to produce robust density maps in a few-shot setting. The method demonstrates significant improvements over state-of-the-art on London, Jiangsu, and Yosemite datasets across six cross-domain tasks, including notable reductions in counting error and boosts in localization accuracy (e.g., Yosemite→Jiangsu), highlighting strong generalization to diverse landscapes. The approach offers a practical path to accurate tree counting in varied environments and can be extended to other object-counting tasks in remote sensing and urban analysis.

Abstract

The process of estimating and counting tree density using only a single aerial or satellite image is a difficult task in the fields of photogrammetry and remote sensing. However, it plays a crucial role in the management of forests. The huge variety of trees in varied topography severely hinders tree counting models to perform well. The purpose of this paper is to propose a framework that is learnt from the source domain with sufficient labeled trees and is adapted to the target domain with only a limited number of labeled trees. Our method, termed as AdaTreeFormer, contains one shared encoder with a hierarchical feature extraction scheme to extract robust features from the source and target domains. It also consists of three subnets: two for extracting self-domain attention maps from source and target domains respectively and one for extracting cross-domain attention maps. For the latter, an attention-to-adapt mechanism is introduced to distill relevant information from different domains while generating tree density maps; a hierarchical cross-domain feature alignment scheme is proposed that progressively aligns the features from the source and target domains. We also adopt adversarial learning into the framework to further reduce the gap between source and target domains. Our AdaTreeFormer is evaluated on six designed domain adaptation tasks using three tree counting datasets, \ie Jiangsu, Yosemite, and London. Experimental results show that AdaTreeFormer significantly surpasses the state of the art, \eg in the cross domain from the Yosemite to Jiangsu dataset, it achieves a reduction of 15.9 points in terms of the absolute counting errors and an increase of 10.8\% in the accuracy of the detected trees' locations. The codes and datasets are available at https://github.com/HAAClassic/AdaTreeFormer.

Figures (9)

• Figure 1: Overview of the proposed AdaTreeFormer framework. (a) In the source domain subnet, the estimated tree density maps of the source domain are optimized with ground truth (GT) using $L_{TDM}$. The similar pipeline goes to the target domain subnet with the few-shot target domain images, obtained through Cutmix augmentation. The estimated tree density maps in the source-target subnet, produced after the cross-domain attention, are optimized using GT of the target domain through $L_{TDM}$, while the estimated feature maps are refined using $L_{DT}$ and $L_{DS}$. An adversarial training is employed that is optimized with $L_{Adv}$. (b) Explanation of the processes of advised adversarial training in one iteration. (c) The structure of the HCDFA using the self- and cross-domain attention score maps ($A_{SDASM}$ and $A_{CDASM}$) for a specific scale. When the input feature maps in the lower yellow part are from the source subnet, the equality between $A_{SDASM}$ and $A_{CDASM}$ is equivalent to $L_{DS}$ in $L_{HCDFA}$, and when those feature maps are from the target subnet, this equality is equal to $L_{DT}$.
• Figure 2: (a) The details of the encoder-decoder part of the proposed AdaTreeFormer. Given the input image, multi-scale features are firstly extracted through the linear embedding, shifted windows transformer block (SWTB), and patch merging module in the encoder. The domain attention blocks (DAB) and density estimation block (DEB) in the decoder align the source and target domains and generate the tree density map, respectively.
• Figure 3: (a) The structure of the SWTB for extracting feature maps. (b) The patch merging module incorporates information from different image patches into a single unified representation.
• Figure 4: (a) The structure of the DAB that generates the self- or cross-domain attention maps using the produced feature maps from the encoder part of the network, (b) The DEB estimates the final tree density map by fusing the generated feature maps from the encoder ($\mathcal{F}_{1,2}^s$ and $\mathcal{F}_{1,2}^t$) and the last layer of the decoder ($A_{attn}^{'}$).
• Figure 5: Some samples of RGB images and corresponding tree density maps of the Yosemite, London, and Jiangsu datasets.