Source-free Semantic Regularization Learning for Semi-supervised Domain Adaptation

TL;DR

SERL tackles semi-supervised domain adaptation in a source-free setting by learning target semantic information through semantic regularization. It introduces three regularization techniques: SPCR, HMR, and TPR, to capture target semantics from probabilistic, hard-sample, and prediction-regularization perspectives. Empirical results on DomainNet, Office-Home, and Office-31 show state-of-the-art performance, with notable gains in 1-shot and 3-shot settings. The work highlights the importance of explicit target semantic learning for robust cross-domain transfer and demonstrates the practicality of a source-free SSDA regime.

Abstract

Semi-supervised domain adaptation (SSDA) has been extensively researched due to its ability to improve classification performance and generalization ability of models by using a small amount of labeled data on the target domain. However, existing methods cannot effectively adapt to the target domain due to difficulty in fully learning rich and complex target semantic information and relationships. In this paper, we propose a novel SSDA learning framework called semantic regularization learning (SERL), which captures the target semantic information from multiple perspectives of regularization learning to achieve adaptive fine-tuning of the source pre-trained model on the target domain. SERL includes three robust semantic regularization techniques. Firstly, semantic probability contrastive regularization (SPCR) helps the model learn more discriminative feature representations from a probabilistic perspective, using semantic information on the target domain to understand the similarities and differences between samples. Additionally, adaptive weights in SPCR can help the model learn the semantic distribution correctly through the probabilities of different samples. To further comprehensively understand the target semantic distribution, we introduce hard-sample mixup regularization (HMR), which uses easy samples as guidance to mine the latent target knowledge contained in hard samples, thereby learning more complete and complex target semantic knowledge. Finally, target prediction regularization (TPR) regularizes the target predictions of the model by maximizing the correlation between the current prediction and the past learned objective, thereby mitigating the misleading of semantic information caused by erroneous pseudo-labels. Extensive experiments on three benchmark datasets demonstrate that our SERL method achieves state-of-the-art performance.

Figures (10)

• Figure 1: The learning scenario of our SERL framework. Different from the training paradigm of most existing SSDA methods, we adopt a source-free training strategy. The source model comprises a feature extractor and a classifier initialized on the source domain. We focus on improving the target domain adaptation stage of the model. In the target domain adaptation stage, SERL freezes the classifier module and fine-tunes the feature extractor module through semantic regularization learning.
• Figure 2: The motivation of our SERL. (a) Due to the scarcity of target semantic labels during training, most existing SSDA methods have shortcomings in target semantic learning, resulting in models only learning limited knowledge (e.g., only the relationships between samples) on the target domain. When more complex relationships exist on the target domain, such as hard and noisy samples, the model may perform poorly due to a lack of understanding of semantic information. (b) Our SERL utilizes the semantic information learned on the target domain from the perspective of semantic regularization to constrain the feature representation of the model further, thereby adapting to more complex target domain distributions.
• Figure 3: Illustration of our proposed semantic regularization learning (SERL) framework. Left: The model initialized on the source domain is adaptively fine-tuned on the target domain. The labeled target data and the strong and weak augmented versions of the unlabeled target data are input to the feature extractor $g$, then sent to the classifier $f$, and further learned the target domain knowledge through semantic regularization. The two feature extractors and classifiers used share parameter weight. Right: (a) Semantic probability contrastive regularization (SPCR) adaptively learns discriminative features through target semantic information and helps the model obtain a more confident probability output. (b) Hard-sample mixup regularization (HMR) uses the semantic information of easy samples to guide the model in learning the distribution of hard target samples, helping the model learn more complex target domain distributions. (c) Target prediction regularization (TPR) is used to minimize the misleading of erroneous semantic information to the model from the perspective of noise labels and help the model learn the true target domain distribution information.
• Figure 4: The impact of source-free learning frameworks on performance. The experiments were conducted in three scenarios of Office-Home under the 1-shot setting. SF stands for source-free training paradigm, and LP stands for label propagation.
• Figure 5: The effect of different loss balance parameters $\lambda_\mathrm{prob}$, $\lambda_\mathrm{mix}$, and $\lambda_\mathrm{pre}$ on the model classification accuracy in the Office-Home C$\rightarrow$A and DomainNet R$\rightarrow$C scenario under the 1-shot setting.