Soft-Masked Semi-Dual Optimal Transport for Partial Domain Adaptation

TL;DR

This work tackles Partial Domain Adaptation where the target label space is a subset of the source. It introduces Soft-masked Semi-Dual Optimal Transport (SSOT), a non-adversarial OT framework that uses class-importance weights and a soft label-informed mask to achieve fine-grained, class-wise domain alignment in the shared label space. By employing the entropy-regularized semi-dual OT and neural network parameterization of the Kantorovich potential, SSOT scales to large datasets and integrates with end-to-end feature learning. Empirical results on four benchmarks show that SSOT outperforms state-of-the-art PDA methods, with ablations and visualizations confirming the contributions of the weighting, masking, and OT-based alignment. Overall, SSOT provides a scalable, effective approach to mitigate negative transfer due to label shift in PDA and promotes discriminative, transferable representations.

Abstract

Visual domain adaptation aims to learn discriminative and domain-invariant representation for an unlabeled target domain by leveraging knowledge from a labeled source domain. Partial domain adaptation (PDA) is a general and practical scenario in which the target label space is a subset of the source one. The challenges of PDA exist due to not only domain shift but also the non-identical label spaces of domains. In this paper, a Soft-masked Semi-dual Optimal Transport (SSOT) method is proposed to deal with the PDA problem. Specifically, the class weights of domains are estimated, and then a reweighed source domain is constructed, which is favorable in conducting class-conditional distribution matching with the target domain. A soft-masked transport distance matrix is constructed by category predictions, which will enhance the class-oriented representation ability of optimal transport in the shared feature space. To deal with large-scale optimal transport problems, the semi-dual formulation of the entropy-regularized Kantorovich problem is employed since it can be optimized by gradient-based algorithms. Further, a neural network is exploited to approximate the Kantorovich potential due to its strong fitting ability. This network parametrization also allows the generalization of the dual variable outside the supports of the input distribution. The SSOT model is built upon neural networks, which can be optimized alternately in an end-to-end manner. Extensive experiments are conducted on four benchmark datasets to demonstrate the effectiveness of SSOT.

Figures (7)

• Figure 1: Illustration of the PDA problem. UDA assumes the source and target domains share the same label space, i.e., $\mathcal{Y}^t\!=\!\mathcal{Y}^s$. Direct application of the classifier learned on the source domain suffers from domain shift, as shown in the left. Compared with UDA, PDA assumes that the label space of the target domain is a subset of the source domain, i.e., $\mathcal{Y}^t\!\subset \!\mathcal{Y}^s$. The challenges of PDA are not only from domain shift but also the mismatch of the label spaces. Best viewed in color.
• Figure 2: Flowchart of the proposed SSOT for PDA. The source and target domains share the network weights of the feature extractor $f(\cdot)$. To identify the source outlier classes and mitigate the label shift across domain, an importance weight $\boldsymbol{m}$ is introduced to reweigh the source domain. Then, the semi-dual OT formulation reduces the distribution discrepancy between the reweighed source domain $P_X^{\tilde{s}}$ and the target domain $P_X^t$. Besides, the cost matrix is enhanced by a soft-mask matrix to capture more class-relevant structures across domains. Specifically, the OT solver is parameterized by a neural network $g(\cdot)$ to approximate the Kantorovich potential. Best viewed in color.
• Figure 3: Hyper-parameter sensitivity of $\lambda_{\rm Ent}$ and $\lambda_{\rm OT}$ on ImageCLEF and Office-31. Best viewed in color.
• Figure 4: The t-SNE visualizations of features generated by Source, PADA, AR, and SSOT on Image-CLEF task P$\rightarrow$I and Office-31 task A$\rightarrow$D, respectively. Here, "o" means source domain, and "+" means target domain. Each color denotes one class. Best viewed in color.
• Figure 5: Ablation study on Image-CLEF task P$\rightarrow$I and Office-31 task A$\rightarrow$D.