Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantic Bridging Domains: Pseudo-Source as Test-Time Connector

Xizhong Yang, Huiming Wang, Ning Xu, Mofei Song

TL;DR

This work proposes a Stepwise Semantic Alignment (SSA) method, viewing the pseudo-source as a semantic bridge connecting the source and target, rather than a direct substitute for the source, to enhance the semantic quality of the SSA process in the absence of source and ground truth of target domains.

Abstract

Distribution shifts between training and testing data are a critical bottleneck limiting the practical utility of models, especially in real-world test-time scenarios. To adapt models when the source domain is unknown and the target domain is unlabeled, previous works constructed pseudo-source domains via data generation and translation, then aligned the target domain with them. However, significant discrepancies exist between the pseudo-source and the original source domain, leading to potential divergence when correcting the target directly. From this perspective, we propose a Stepwise Semantic Alignment (SSA) method, viewing the pseudo-source as a semantic bridge connecting the source and target, rather than a direct substitute for the source. Specifically, we leverage easily accessible universal semantics to rectify the semantic features of the pseudo-source, and then align the target domain using the corrected pseudo-source semantics. Additionally, we introduce a Hierarchical Feature Aggregation (HFA) module and a Confidence-Aware Complementary Learning (CACL) strategy to enhance the semantic quality of the SSA process in the absence of source and ground truth of target domains. We evaluated our approach on tasks like semantic segmentation and image classification, achieving a 5.2% performance boost on GTA2Cityscapes over the state-of-the-art.

Semantic Bridging Domains: Pseudo-Source as Test-Time Connector

TL;DR

This work proposes a Stepwise Semantic Alignment (SSA) method, viewing the pseudo-source as a semantic bridge connecting the source and target, rather than a direct substitute for the source, to enhance the semantic quality of the SSA process in the absence of source and ground truth of target domains.

Abstract

Distribution shifts between training and testing data are a critical bottleneck limiting the practical utility of models, especially in real-world test-time scenarios. To adapt models when the source domain is unknown and the target domain is unlabeled, previous works constructed pseudo-source domains via data generation and translation, then aligned the target domain with them. However, significant discrepancies exist between the pseudo-source and the original source domain, leading to potential divergence when correcting the target directly. From this perspective, we propose a Stepwise Semantic Alignment (SSA) method, viewing the pseudo-source as a semantic bridge connecting the source and target, rather than a direct substitute for the source. Specifically, we leverage easily accessible universal semantics to rectify the semantic features of the pseudo-source, and then align the target domain using the corrected pseudo-source semantics. Additionally, we introduce a Hierarchical Feature Aggregation (HFA) module and a Confidence-Aware Complementary Learning (CACL) strategy to enhance the semantic quality of the SSA process in the absence of source and ground truth of target domains. We evaluated our approach on tasks like semantic segmentation and image classification, achieving a 5.2% performance boost on GTA2Cityscapes over the state-of-the-art.
Paper Structure (56 sections, 1 theorem, 34 equations, 9 figures, 14 tables, 1 algorithm)

This paper contains 56 sections, 1 theorem, 34 equations, 9 figures, 14 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Source Distribution Estimation for TTA.
  4. Pseudo Labeling for TTA.
  5. Methodology
  6. Problem definition
  7. Overview
  8. Stepwise Semantic Alignment
  9. Data Selection.
  10. Remaining-Target Semantic Alignment.
  11. Hierarchical Feature Aggregation
  12. Confidence-Aware Complementary Learning
  13. Experiments
  14. Datasets and evaluation mtrics
  15. Implementation details
  16. ...and 41 more sections

Key Result

Theorem 3.1

Let $\boldsymbol{p} \in \Delta^{C-1}$ be a categorical distribution over label space $\mathcal{Y} = \{1, \dots, C\}$, with entropy bounded by $\mathcal{H}(\boldsymbol{p}) \le H_0$. Then for any $\alpha \in (0,1)$, there exist thresholds $\tau_\alpha > \tau_\beta$ such that: satisfying:

Figures (9)

  • Figure 1: Framework of Stepwise Semantic Alignment (SSA). Source Model* and Pre-Trained Model* are their respective feature extractors. TD-S and PRE-S refer to the Semantic features derived from the Target-Domain (idealized semantics) and the PRE-Trained model, respectively. PS-S and RT-S correspond to the Semantic features extracted from $D_{ps}$ and $D_{rt}$. See \ref{['sec:appendix_pseudo_code']} for pseudo-code.
  • Figure 2: SSA segmentation visualization on GTA5$\to$Cityscapes.
  • Figure 3: SSA t-SNE visualization on VisDA-C.
  • Figure 4: Scaling effect of SSA's gains in different tasks.
  • Figure 5: Sensitivity analysis for the thresholds.
  • ...and 4 more figures

Theorems & Definitions (3)

  • Theorem 3.1
  • proof
  • proof