A Survey of Unsupervised Deep Domain Adaptation

TL;DR

Unsupervised deep domain adaptation tackles performance drop caused by domain shift by leveraging labeled source data and unlabeled target data. The paper surveys a broad taxonomy of methods, including domain-invariant feature learning, domain mapping via image translation, normalization-based alignment, ensembles, and target-discriminative strategies, plus their combinations, losses, training regimes, and network architectures. It discusses theoretical bounds linking source error, domain divergence, and target error under both unsupervised and semi-supervised settings, and surveys applications across computer vision, NLP, and time series. The findings highlight that no single approach universally dominates and that combining complementary components, data augmentation, and careful hyperparameter tuning can yield strong performance, with future directions including bi-directional adaptation and richer evaluation. The work provides a structured framework and benchmarks to guide researchers in designing and comparing unsupervised deep domain adaptation methods.

Abstract

Deep learning has produced state-of-the-art results for a variety of tasks. While such approaches for supervised learning have performed well, they assume that training and testing data are drawn from the same distribution, which may not always be the case. As a complement to this challenge, single-source unsupervised domain adaptation can handle situations where a network is trained on labeled data from a source domain and unlabeled data from a related but different target domain with the goal of performing well at test-time on the target domain. Many single-source and typically homogeneous unsupervised deep domain adaptation approaches have thus been developed, combining the powerful, hierarchical representations from deep learning with domain adaptation to reduce reliance on potentially-costly target data labels. This survey will compare these approaches by examining alternative methods, the unique and common elements, results, and theoretical insights. We follow this with a look at application areas and open research directions.

Figures (6)

• Figure 1: Realistic but entirely synthetic images of human faces generated by a GAN trained on the CelebA-HQ dataset karras2018progressive.
• Figure 2: Illustration of the GAN generator $G$ and discriminator $D$ networks. The dashed line between the $D$ networks indicates that they share weights (or are the same network). In the top row, a real image from the training data (horses $\leftrightarrow$ zebras dataset by Zhu et al. zhu2017iccv) is fed to the discriminator, and the goal of $D$ is to make $D(x)=1$ (correctly classify as real). In the bottom row, a fake image from the generator is fed to the discriminator, and the goal of $D$ is to make $D(G(z))=0$ (correctly classify as fake), which competes with the goal of $G$ to make $D(G(z))=1$ (misclassify as real).
• Figure 3: General network setup for domain adaptation methods learning domain-invariant features. (a) Methods differ in regard to how the domains are aligned during training (the Alignment Component) and whether the feature extractors used on each domain share none, some, or all of the weights between domains. (b) The target data are fed to the domain-invariant feature extractor and then to the task classifier.
• Figure 4: Synthetic vehicle driving image (left) adapted to look realistic (right) hoffman2018icml.
• Figure 5: Two possible configurations using image-to-image translation for domain adaptation. The conditional GAN and classifier can be trained separately or jointly. Method 1 is the most common. Method 2 is used by one paper. A combination of methods 1 and 2 is used in one paper. The dashed lines between networks indicate that they share weights (or are the same network). Note: this figure does not illustrate the many variants of the conditional GAN component, which often train a generator in each direction (one source to target and one target to source) and use additional losses such as cycle consistency.