Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation

TL;DR

Self-Training with Dynamic Weighting (STDW) tackles robust Gradual Domain Adaptation by introducing a time-varying balance parameter $\varrho$ that progressively shifts learning focus from the source to the target domain within a self-training loop. It couples dynamic pseudo-labeling, cyclic batch matching, and stepwise osmosis to enable stable, progressive knowledge migration across intermediate domains. Empirical results on Rotated MNIST, Color-Shift MNIST, Portraits, and Cover Type, as well as corruption benchmarks CIFAR-10-C and CIFAR-100-C, show state-of-the-art performance and strong robustness to distributional shifts; ablations confirm the critical role of the $\varrho$ schedule and intermediate domains. The approach provides both theoretical stability insights and a practical framework for deploying reliable gradual domain adaptation in dynamic real-world settings.

Abstract

In this paper, we propose a new method called Self-Training with Dynamic Weighting (STDW), which aims to enhance robustness in Gradual Domain Adaptation (GDA) by addressing the challenge of smooth knowledge migration from the source to the target domain. Traditional GDA methods mitigate domain shift through intermediate domains and self-training but often suffer from inefficient knowledge migration or incomplete intermediate data. Our approach introduces a dynamic weighting mechanism that adaptively balances the loss contributions of the source and target domains during training. Specifically, we design an optimization framework governed by a time-varying hyperparameter $\varrho$ (progressing from 0 to 1), which controls the strength of domain-specific learning and ensures stable adaptation. The method leverages self-training to generate pseudo-labels and optimizes a weighted objective function for iterative model updates, maintaining robustness across intermediate domains. Experiments on rotated MNIST, color-shifted MNIST, portrait datasets, and the Cover Type dataset demonstrate that STDW outperforms existing baselines. Ablation studies further validate the critical role of $\varrho$'s dynamic scheduling in achieving progressive adaptation, confirming its effectiveness in reducing domain bias and improving generalization. This work provides both theoretical insights and a practical framework for robust gradual domain adaptation, with potential applications in dynamic real-world scenarios. The code is available at https://github.com/Dramwig/STDW.

Figures (3)

• Figure 1: Overview of gradual domain adaptation. The classifiers perform accurately in classifying the sample points in the current domain and the neighboring domains, but fail when classifying the samples from the Source directly to the Target domain.
• Figure 2: The framework of our Self-Training with Dynamic Weighting (STDW). It is designed to facilitate a smooth and controlled adaptation of a model from a source domain to a target domain. The hyperparameter $\varrho$ governs the trade-off between the two domains: when $\varrho = 0$, the model operates exclusively on the source domain, whereas as $\varrho$ increases toward 1, the model progressively shifts its emphasis toward the target domain. Pseudo-labels, generated by the model itself, guide the adaptation process, thereby ensuring a gradual and stable transition across domains.
• Figure 3: Ablation study of our STDW under different domain configurations across four benchmark datasets. The "2-domain" setting includes only the source and target domains (i.e., no intermediate domains), whereas the "multi-domain" settings incorporate additional unlabeled intermediate domains that gradually bridge the source and target.