GDO:Gradual Domain Osmosis

TL;DR

An optimisation framework based on the hyperparameter $\lambda$ is designed by dynamically balancing the loss weights of the source and target domains, which enables the model to progressively adjust the strength of knowledge migration during the training process, thus achieving cross-domain generalisation more efficiently.

Abstract

In this paper, we propose a new method called Gradual Domain Osmosis, which aims to solve the problem of smooth knowledge migration from source domain to target domain in Gradual Domain Adaptation (GDA). Traditional Gradual Domain Adaptation methods mitigate domain bias by introducing intermediate domains and self-training strategies, but often face the challenges of inefficient knowledge migration or missing data in intermediate domains. In this paper, we design an optimisation framework based on the hyperparameter $λ$ by dynamically balancing the loss weights of the source and target domains, which enables the model to progressively adjust the strength of knowledge migration ($λ$ incrementing from 0 to 1) during the training process, thus achieving cross-domain generalisation more efficiently. Specifically, the method incorporates self-training to generate pseudo-labels and iteratively updates the model by minimising a weighted loss function to ensure stability and robustness during progressive adaptation in the intermediate domain. The experimental part validates the effectiveness of the method on rotated MNIST, colour-shifted MNIST, portrait dataset and forest cover type dataset, and the results show that it outperforms existing baseline methods. The paper further analyses the impact of the dynamic tuning strategy of the hyperparameter $λ$ on the performance through ablation experiments, confirming the advantages of progressive domain penetration in mitigating the domain bias and enhancing the model generalisation capability. The study provides a theoretical support and practical framework for asymptotic domain adaptation and expands its application potential in dynamic environments.

Figures (4)

• Figure 1: An example of domain shift. The red is source domain and the blue is target domain. The yellow regions are assumed to have zero support in the target distribution.
• Figure 2: The framework of our GDO. It is desiqned to adapt a model from a source domain to a target domain in a smooth and controlled manner. The hyperparameter $\lambda$ controls the trade-off between the two domains: when $\lambda=0$, the model focuses entirely on the source domain, and as $\lambda$ increases to 1, the model progressively shifts its focus to the target domain. Pseudo-labels, generated by the model itself, guide the learning process, ensuring gradual and stable adaptation.
• Figure 3: The process of Gradual Domain Adaptation with Dynamic Pseudo-Labeling and Self-Training. The diagram demonstrates the generation of hard pseudo-labels at each stage of the domain adaptation process. The classifiers $C^{(t,k)}$ are progressively updated, using a dual-timescale update rule for intra-domain and inter-domain adaptation. The model updates the feature extractor parameters $\theta$ in a high-frequency manner and classifier head parameters $\phi$ in a low-frequency manner to facilitate smooth domain transitions. This approach promotes rapid domain adaptation and minimizes the loss between predictions and pseudo-labels across the domains $\mathcal{D}_t$ and $\mathcal{D}_{t+1}$.
• Figure 4: Ablation experiments with different domain conditions are conducted on four datasets. Two domains represent no intermediate domains (only source domain and target domain), and the added domains are all intermediate domains.