Learning from Different Samples: A Source-free Framework for Semi-supervised Domain Adaptation

TL;DR

A novel source-free framework to achieve semi-supervised fine-tuning of the source pre-trained model on the target domain, and decouples SSDA from the perspectives of different target samples, specifically designing robust learning techniques for unlabeled, reliably labeled, and noisy pseudo-labeled target samples.

Abstract

Semi-supervised domain adaptation (SSDA) has been widely studied due to its ability to utilize a few labeled target data to improve the generalization ability of the model. However, existing methods only consider designing certain strategies for target samples to adapt, ignoring the exploration of customized learning for different target samples. When the model encounters complex target distribution, existing methods will perform limited due to the inability to clearly and comprehensively learn the knowledge of multiple types of target samples. To fill this gap, this paper focuses on designing a framework to use different strategies for comprehensively mining different target samples. We propose a novel source-free framework (SOUF) to achieve semi-supervised fine-tuning of the source pre-trained model on the target domain. Different from existing SSDA methods, SOUF decouples SSDA from the perspectives of different target samples, specifically designing robust learning techniques for unlabeled, reliably labeled, and noisy pseudo-labeled target samples. For unlabeled target samples, probability-based weighted contrastive learning (PWC) helps the model learn more discriminative feature representations. To mine the latent knowledge of labeled target samples, reliability-based mixup contrastive learning (RMC) learns complex knowledge from the constructed reliable sample set. Finally, predictive regularization learning (PR) further mitigates the misleading effect of noisy pseudo-labeled samples on the model. Extensive experiments on benchmark datasets demonstrate the superiority of our framework over state-of-the-art methods.

Figures (3)

• Figure 1: Differences between the proposed method and existing methods. Figures (a) and (b) show that our method focuses on fine-tuning in a semi-supervised target domain without the source data. Based on this learning framework, SOUF performs customized learning for different types of target samples. Figure (c) shows that our method achieves SoTA with a large advantage on the DomainNet and Office-Home datasets.
• Figure 2: Illustration of our source-free framework. For unlabeled samples, probability-weighted contrastive learning (PWC) adaptively learns discriminative features, enhancing credible probability outputs. To further exploit labeled samples, reliability-based mixup contrastive learning (RMC) mixes patches from reliable samples, aiding in complex representation learning. Predictive regularization (PR) minimizes the impact of erroneous semantic information from noisy labels.
• Figure 3: The effect of different loss balance parameters $\lambda_\mathrm{pwc}$, $\lambda_\mathrm{rmc}$, and $\lambda_\mathrm{pr}$ on the model classification accuracy in the Office-Home P$\rightarrow$C scenario under the 1-shot setting.