Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Face Generation and Editing with StyleGAN: A Survey

Andrew Melnik, Maksim Miasayedzenkau, Dzianis Makarovets, Dzianis Pirshtuk, Eren Akbulut, Dennis Holzmann, Tarek Renusch, Gustav Reichert, Helge Ritter

TL;DR

This survey covers the evolution of StyleGAN, from PGGAN to StyleGAN3, and explores relevant topics such as suitable metrics for training, different latent representations, GAN inversion to latent spaces of StyleGAN, face image editing, cross-domain face stylization, face restoration, and even Deepfake applications.

Abstract

Our goal with this survey is to provide an overview of the state of the art deep learning methods for face generation and editing using StyleGAN. The survey covers the evolution of StyleGAN, from PGGAN to StyleGAN3, and explores relevant topics such as suitable metrics for training, different latent representations, GAN inversion to latent spaces of StyleGAN, face image editing, cross-domain face stylization, face restoration, and even Deepfake applications. We aim to provide an entry point into the field for readers that have basic knowledge about the field of deep learning and are looking for an accessible introduction and overview.

Face Generation and Editing with StyleGAN: A Survey

TL;DR

This survey covers the evolution of StyleGAN, from PGGAN to StyleGAN3, and explores relevant topics such as suitable metrics for training, different latent representations, GAN inversion to latent spaces of StyleGAN, face image editing, cross-domain face stylization, face restoration, and even Deepfake applications.

Abstract

Our goal with this survey is to provide an overview of the state of the art deep learning methods for face generation and editing using StyleGAN. The survey covers the evolution of StyleGAN, from PGGAN to StyleGAN3, and explores relevant topics such as suitable metrics for training, different latent representations, GAN inversion to latent spaces of StyleGAN, face image editing, cross-domain face stylization, face restoration, and even Deepfake applications. We aim to provide an entry point into the field for readers that have basic knowledge about the field of deep learning and are looking for an accessible introduction and overview.
Paper Structure (64 sections, 4 equations, 23 figures)

This paper contains 64 sections, 4 equations, 23 figures.

Table of Contents

  1. Introduction
  2. Survey Overview
  3. Synopsis of StyleGAN Applications
  4. Training datasets
  5. StyleGAN Architectures for Generation of Faces
  6. Generative Adversarial Networks
  7. Progressive growing GANs (PGGAN)
  8. StyleGAN
  9. StyleGAN2
  10. StyleGAN3
  11. Measuring Similarity of Faces
  12. Adversarial Loss
  13. $L_2$ Loss
  14. LPIPS Loss
  15. Identity Preservation Loss
  16. ...and 49 more sections

Figures (23)

  • Figure 1: Synopsis of StyleGAN Applications. A. Faces generated using StyleGAN2 karras2020analyzing. B. NFT collection GANFolksAll generated using StyleGAN2 karras2020analyzing trained on MetFaces dataset karras2020training. C. Style mixing (see Figure \ref{['fig:style_mixing']}). D. From left to right: the source image, smile removed, gender changed karras2020analyzing. E. Image editing with StyleCLIP patashnik2021styleclip using text prompts. Upper row: original images, lower row: edited images using text prompts -- Emma Stone (left), Mohawk Style (right). F. Seamless image-crossover abdal2020image2stylegan++. Left: sources image, right: resulting image. G. Blind face restoration wang2021towards. Left: degraded image, right: enhanced image. H. Identity preserving face restoration and in-painting nitzan2022mystylemystyleex. I and J. Transferring faces into other domains while preserving the identity. Left: real image, right: cartoon like output ToonifyDBLP:journals/corr/abs-2108-00946K. Control semantic parameters, such as face pose using StyleGAN component. Left: source image, right: pose changed tewari2020stylerig. L. Deepfake generation Zhu2021OneSF. M. Automated 3D avatar generation using StyleGAN component. From left to right: source image, avatar, face model luo2021normalized.
  • Figure 2: (a) StyleGAN grows progressively while training, (b) StyleGAN2 does not but uses output skips and residual connections. Images from Karras2018ProgressiveGONA, karras2020analyzing
  • Figure 3: Style mixing in StyleGANkarras2019style. Top panel shows examples of style mixing, bottom panel illustrates the style mixing pipeline in StyleGAN - latent representations of two images can be used in different levels of the generator.
  • Figure 4: Architectures of StyleGAN generators: (a) StyleGAN karras2019style, (b) StyleGAN2 karras2020analyzing, (c) StyleGAN3 karras2021alias
  • Figure 5: Latent spaces in StyleGAN. Image from bermano2022state.
  • ...and 18 more figures