Low Saturation Confidence Distribution-based Test-Time Adaptation for Cross-Domain Remote Sensing Image Classification

TL;DR

This work tackles cross-domain remote sensing image classification under distribution shift by proposing LSCD-TTA, a source-free, test-time adaptation framework that updates a source-trained network on unlabeled target data during inference. It introduces three RS-tailored losses—Weak-Confidence Softmax-Entropy ($\\mathcal{L}_{wcse}$), Balanced-Categories Softmax-Entropy ($\\mathcal{L}_{bcse}$), and Low Saturation Distribution ($\\mathcal{L}_{lsd}$)—and updates BN statistics online, combining them into $\\mathcal{L}_{lscd}=\\alpha\\mathcal{L}_{wcse}+\\beta\\mathcal{L}_{bcse}+\\tau\\mathcal{L}_{lsd}$. Extensive experiments on six domain-adaptation tasks across AID, NWPU-RESISC45, and UC Merced show that LSCD-TTA outperforms state-of-the-art SFDA and TTA methods, with average accuracy gains of $4.99\\%$ (ResNet-50), $5.22\\%$ (ResNet-101), and $2.37\\%$ (ViT-B/16), while maintaining real-time efficiency. The approach demonstrates robust, rapid adaptation suitable for pragmatic RS applications and offers insights into handling class imbalance, intra-class diversity, and low-confidence samples during test-time adaptation.

Abstract

Unsupervised Domain Adaptation (UDA) has emerged as a powerful technique for addressing the distribution shift across various Remote Sensing (RS) applications. However, most UDA approaches require access to source data, which may be infeasible due to data privacy or transmission constraints. Source-free Domain Adaptation addresses the absence of source data but usually demands a large amount of target domain data beforehand, hindering rapid adaptation and restricting their applicability in broader scenarios. In practical cross-domain RS image classification, achieving a balance between adaptation speed and accuracy is crucial. Therefore, we propose Low Saturation Confidence Distribution Test-Time Adaptation (LSCD-TTA), marketing the first attempt to explore Test-Time Adaptation for cross-domain RS image classification without requiring source or target training data. LSCD-TTA adapts a source-trained model on the fly using only the target test data encountered during inference, enabling immediate and efficient adaptation while maintaining high accuracy. Specifically, LSCD-TTA incorporates three optimization strategies tailored to the distribution characteristics of RS images. Firstly, weak-confidence softmax-entropy loss emphasizes categories that are more difficult to classify to address unbalanced class distribution. Secondly, balanced-categories softmax-entropy loss softens and balances the predicted probabilities to tackle the category diversity. Finally, low saturation distribution loss utilizes soft log-likelihood ratios to reduce the impact of low-confidence samples in the later stages of adaptation. By effectively combining these losses, LSCD-TTA enables rapid and accurate adaptation to the target domain for RS image classification.

Figures (6)

• Figure 1: Comparisons of different domain adaptation methods for remote sensing image classification.
• Figure 2: The main structure of LSCD-TTA. The model $F_{\theta}$ is first pre-trained on the source domain, which serves as the initialization of $G_{\theta}$ for adaptation toward the target domain. The model extracts the features of target test images within the continuous input and calculates the probability distribution using model $G_{\theta}$ during adaptation. Subsequently, LSCD-TTA balances the predicted probabilities across categories (i.e.$\mathcal{L}_{bcse}$) and especially emphasizes low-saturation or week-probability samples that are difficult to distinguish (i.e.$\mathcal{L}_{lsd}$ and $\mathcal{L}_{wcse}$), to align the source and target domains in real time.
• Figure 3: Samples of the datasets
• Figure 4: Sensitivity analysis of $\alpha$, $\beta$, $\tau$ with Resnet-50 (top) and Resnet-101 (bottom)
• Figure 5: Grad-CAM comparison between LSCD-TTA and the baseline model using ResNet-50. The heatmaps illustrate the areas of focus during different adaptation tasks. The baseline model’s attention is constrained by limited source knowledge, often leading to incorrect focus (e.g., misidentifying commercial buildings as storage tanks in U$\rightarrow$A) or misclassification despite correct attention areas (e.g., misclassifying ports as parking lots in U$\rightarrow$A). In contrast, LSCD-TTA enhances attention performance and enables fine-grained classification while preserving global semantic information.