Image-to-Image Translation with Disentangled Latent Vectors for Face Editing

TL;DR

This work addresses the challenge of controllable, disentangled face attribute editing by learning per-attribute latent directions within a deep encoder–decoder framework. It introduces a latent-space translation module with orthogonality and disentanglement losses, coupled with attention-based skip connections to preserve details while enabling targeted edits via $T(e, \alpha, i) = e + \alpha \times A_i$. The proposed VecGAN++ demonstrates strong quantitative and qualitative gains over both end-to-end and StyleGAN inversion-based methods across global and local edits, while offering interpretable control over editing strength. The approach facilitates faithful reconstruction, robust attribute manipulation, and faithful background preservation, with thorough ablations highlighting the contributions of the disentanglement loss and attention skip design.

Abstract

We propose an image-to-image translation framework for facial attribute editing with disentangled interpretable latent directions. Facial attribute editing task faces the challenges of targeted attribute editing with controllable strength and disentanglement in the representations of attributes to preserve the other attributes during edits. For this goal, inspired by the latent space factorization works of fixed pretrained GANs, we design the attribute editing by latent space factorization, and for each attribute, we learn a linear direction that is orthogonal to the others. We train these directions with orthogonality constraints and disentanglement losses. To project images to semantically organized latent spaces, we set an encoder-decoder architecture with attention-based skip connections. We extensively compare with previous image translation algorithms and editing with pretrained GAN works. Our extensive experiments show that our method significantly improves over the state-of-the-arts.

Figures (9)

• Figure 1: VecGAN++ image translation results.
• Figure 2: Our translator is built on the idea of interpretable latent directions. We encode images with an Encoder to a latent representation from which we change a selected tag ($i$), e.g. hair color with a learnable direction $A_i$ and a scale $\alpha$. To calculate the scale, we subtract the target style scale from the source style. This operation corresponds to removing an attribute and adding an attribute. To remove the image's attribute, the source style is encoded and projected from the source image. To add the target attribute, the target style scale is sampled from a distribution mapped for the given attribute ($j$), e.g. black, blonde, or encoded and projected from a reference image. We also propose an attention-based skip connection module to transfer selected features without an information bottleneck to the decoder.
• Figure 3: FID curves on smile addition of models trained with and without disentanglement loss through iterations.
• Figure 4: Visualizations of attention masks for different edits.
• Figure 5: Qualitative results of bangs attribute of VecGAN++, VecGAN and HiSD. Given reference images, methods extract reference attributes and edit input images accordingly. VecGAN++ achieves better edit quality compared to VecGAN and preserves the other facial details better. It is important to note that, HiSD learns feature-based local translators, which is a successful approach on local edits, e.g. bangs, eyeglasses, hair color but not smile, age, or gender. Our method achieves comparable visual and better quantitative results than HiSD on this local task and can also achieve global edits.