Asymmetric GANs for Image-to-Image Translation

TL;DR

AsymmetricGAN addresses the mismatch between translation and reconstruction in asymmetric image-to-image translation by employing dual generators of unequal capacity, $G^t$ for translation and $G^r$ for reconstruction, along with a discriminator. The model supports both unsupervised multi-domain translation and supervised hand-gesture translation, using architecture variants (S1–S3) and distinct parameter-sharing strategies to balance performance and complexity. It introduces a suite of losses—color cycle-consistency, multi-scale SSIM, conditional identity preservation, plus domain-classification and task-specific adversarial/pixel/perceptual losses—to stabilize training and improve result fidelity. Across eight datasets and two task settings, AsymmetricGAN demonstrates superior or competitive generation quality and efficiency relative to strong baselines, validating the value of asymmetric design for complex cross-domain translation tasks.

Abstract

Existing models for unsupervised image translation with Generative Adversarial Networks (GANs) can learn the mapping from the source domain to the target domain using a cycle-consistency loss. However, these methods always adopt a symmetric network architecture to learn both forward and backward cycles. Because of the task complexity and cycle input difference between the source and target domains, the inequality in bidirectional forward-backward cycle translations is significant and the amount of information between two domains is different. In this paper, we analyze the limitation of existing symmetric GANs in asymmetric translation tasks, and propose an AsymmetricGAN model with both translation and reconstruction generators of unequal sizes and different parameter-sharing strategy to adapt to the asymmetric need in both unsupervised and supervised image translation tasks. Moreover, the training stage of existing methods has the common problem of model collapse that degrades the quality of the generated images, thus we explore different optimization losses for better training of AsymmetricGAN, making image translation with higher consistency and better stability. Extensive experiments on both supervised and unsupervised generative tasks with 8 datasets show that AsymmetricGAN achieves superior model capacity and better generation performance compared with existing GANs. To the best of our knowledge, we are the first to investigate the asymmetric GAN structure on both unsupervised and supervised image translation tasks.

Figures (10)

• Figure 1: Unsupervised framework of our AsymmetricGAN for multi-domain image-to-image translation. $z_x$ and $z_y$ indicate the category labels of domain $X$ and $Y$, respectively. $G^t$ and $G^r$ are task-specific asymmetric generators. The translation generator $G^t$ translates images from domain $X$ into domain $Y$ and the reconstruction generator $G^r$ receives the generated image $G^t(x, z_y)$ and the original domain label $z_x$ and tries to recover the input image $x$ during the training stage with the proposed objective functions.
• Figure 2: Supervised framework of our AsymmetricGAN for hand gesture-to-gesture translation. $l_x$ and $l_y$ indicate the hand skeletons of images $x$ and $y$, respectively. $G^t$ and $G^r$ are task-specific asymmetric generators. The translation generator $G^t$ converts images from domain $X$ into domain $Y$ and the construction generator $G^r$ receives the generated image $G^t(x, l_y)$ and the original hand skeleton $l_x$ and attempts to reconstruct the original image $x$ during the optimization with the proposed different objective losses. We have two cycles, i.e., $x {\mapsto} y' {\mapsto} \widehat{x} {\approx} x$ and $y {\mapsto} x' {\mapsto} \widehat{y} {\approx} y$, but we only show one here, i.e., $x {\mapsto} y' {\mapsto} \widehat{x} {\approx} x$.
• Figure 3: Different methods for label$\leftrightarrow$photo translation on Facades. From left to right: Input, Ground Truth (GT), CycleGAN zhu2017unpaired, DualGAN yi2017dualgan, ComboGAN anoosheh2017combogan, DistanceGAN benaim2017one, DistanceGAN+Cycle Loss benaim2017one, DistanceGAN+Self Distance benaim2017one, StarGAN choi2017stargan, Pix2pix isola2017image, BicycleGAN zhu2017toward, and AsymmetricGAN (Ours).
• Figure 4: Different methods for multi-domain facial expression translation on AR Face. From left to right: Input, Ground Truth (GT), CycleGAN zhu2017unpaired, DualGAN yi2017dualgan, ComboGAN anoosheh2017combogan, DistanceGAN benaim2017one, DistanceGAN+Cycle Loss benaim2017one, DistanceGAN+Self Distance benaim2017one, StarGAN choi2017stargan, Pix2pix isola2017image, BicycleGAN zhu2017toward, and AsymmetricGAN (Ours).
• Figure 5: Different methods for multi-domain facial expression translation on Bu3dfe. From left to right: Input, Ground Truth (GT), CycleGAN zhu2017unpaired, DualGAN yi2017dualgan, ComboGAN anoosheh2017combogan, DistanceGAN benaim2017one, DistanceGAN+Cycle Loss benaim2017one, DistanceGAN+Self Distance benaim2017one, StarGAN choi2017stargan, Pix2pix isola2017image, BicycleGAN zhu2017toward, and AsymmetricGAN (Ours).