Two stages domain invariant representation learners solve the large co-variate shift in unsupervised domain adaptation with two dimensional data domains
Hisashi Oshima, Tsuyoshi Ishizone, Tomoyuki Higuchi
TL;DR
The paper tackles unsupervised domain adaptation under large two-dimensional covariate shifts by introducing a two-stage, end-to-end domain invariant representation learning framework that leverages an unsupervised intermediate domain to bridge source, intermediate, and target distributions. A key feature is the L_propose loss, decomposing domain alignment across two adjacent domain pairs, which improves convergence and discriminability on the target. The authors also derive an RV-based theorem for unsupervised hyperparameter tuning, enabling parameter selection without target labels. Empirical results across four datasets and 38 tasks show that the proposed method consistently outperforms baselines (Step-by-step and Normal) and adheres to an upper-bound ordering, especially under larger shifts, with both quantitative gains and qualitative evidence of robust domain invariance and target discriminability. The approach is framed as broadly applicable, with potential extensions including synthetic intermediate domains and integration with other UDA techniques, making it practically impactful for real-world transfer learning scenarios.
Abstract
Recent developments in the unsupervised domain adaptation (UDA) enable the unsupervised machine learning (ML) prediction for target data, thus this will accelerate real world applications with ML models such as image recognition tasks in self-driving. Researchers have reported the UDA techniques are not working well under large co-variate shift problems where e.g. supervised source data consists of handwritten digits data in monotone color and unsupervised target data colored digits data from the street view. Thus there is a need for a method to resolve co-variate shift and transfer source labelling rules under this dynamics. We perform two stages domain invariant representation learning to bridge the gap between source and target with semantic intermediate data (unsupervised). The proposed method can learn domain invariant features simultaneously between source and intermediate also intermediate and target. Finally this achieves good domain invariant representation between source and target plus task discriminability owing to source labels. This induction for the gradient descent search greatly eases learning convergence in terms of classification performance for target data even when large co-variate shift. We also derive a theorem for measuring the gap between trained models and unsupervised target labelling rules, which is necessary for the free parameters optimization. Finally we demonstrate that proposing method is superiority to previous UDA methods using 4 representative ML classification datasets including 38 UDA tasks. Our experiment will be a basis for challenging UDA problems with large co-variate shift.