CyCADA: Cycle-Consistent Adversarial Domain Adaptation

TL;DR

CyCADA addresses domain shift by integrating cycle-consistent adversarial translation with semantic constraints, enabling unsupervised adaptation at both pixel and feature levels. It preserves content via cycle losses and enforces semantic consistency through a fixed source classifier, with optional feature-space alignment to further harmonize representations. Tested on digit recognition and urban-scene segmentation, CyCADA achieves state-of-the-art results, notably closing much of the gap in synthetic-to-real segmentation and delivering strong gains in cross-season synthetic data. The approach offers interpretable image-space translations and demonstrates that joint pixel- and feature-level alignment yields complementary benefits for robust cross-domain transfer.

Abstract

Domain adaptation is critical for success in new, unseen environments. Adversarial adaptation models applied in feature spaces discover domain invariant representations, but are difficult to visualize and sometimes fail to capture pixel-level and low-level domain shifts. Recent work has shown that generative adversarial networks combined with cycle-consistency constraints are surprisingly effective at mapping images between domains, even without the use of aligned image pairs. We propose a novel discriminatively-trained Cycle-Consistent Adversarial Domain Adaptation model. CyCADA adapts representations at both the pixel-level and feature-level, enforces cycle-consistency while leveraging a task loss, and does not require aligned pairs. Our model can be applied in a variety of visual recognition and prediction settings. We show new state-of-the-art results across multiple adaptation tasks, including digit classification and semantic segmentation of road scenes demonstrating transfer from synthetic to real world domains.

Figures (10)

• Figure 1: We propose CyCADA, an adversarial unsupervised adaptation algorithm which uses cycle and semantic consistency to perform adaptation at multiple levels in a deep network. Our model provides significant performance improvements over source model baselines.
• Figure 2: Cycle-consistent adversarial adaptation of pixel-space inputs. By directly remapping source training data into the target domain, we remove the low-level differences between the domains, ensuring that our task model is well-conditioned on target data. We depict here the image-level GAN loss (green), the feature level GAN loss (orange), the source and target semantic consistency losses (black), the source cycle loss (red), and the source task loss (purple). For clarity the target cycle is omitted.
• Figure 3: Ablation: Effect of Semantic or Cycle Consistency Examples of translation failures without the semantic consistency loss. Each triple contains the original SVHN image (left), the image translated into MNIST style (middle), and the image reconstructed back into SVHN (right). (a) Without semantic loss, both the GAN and cycle constraints are satisfied (translated image matches MNIST style and reconstructed image matches original), but the image translated to the target domain lacks the proper semantics. (b) Without cycle loss, the reconstruction is not satisfied and though the semantic consistency leads to some successful semantic translations (top) there are still cases of label flipping (bottom).
• Figure 4: Cross Season Image Translation. Example image-space conversions for the SYNTHIA seasons adaptation setting. We show real samples from each domain (Fall and Winter) alongside conversions to the opposite domain.
• Figure 5: GTA5 to CityScapes Semantic Segmentation. Each test CityScapes image (a) along with the corresponding predictions from the source only model (b) and our CyCADA model (c) are shown and may be compared against the ground truth annotation (d).