Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Unifying Generator Loss Function for Generative Adversarial Networks

Justin Veiner, Fady Alajaji, Bahman Gharesifard

TL;DR

It is shown that this Lα-GAN problem recovers as special cases a number of GAN problems in the literature, including VanillaGAN, least squares GAN (LSGAN), least kth-order GAN (LkGAN), and the recently introduced (αD,αG)-GAN with αD=1.

Abstract

A unifying $α$-parametrized generator loss function is introduced for a dual-objective generative adversarial network (GAN), which uses a canonical (or classical) discriminator loss function such as the one in the original GAN (VanillaGAN) system. The generator loss function is based on a symmetric class probability estimation type function, $\mathcal{L}_α$, and the resulting GAN system is termed $\mathcal{L}_α$-GAN. Under an optimal discriminator, it is shown that the generator's optimization problem consists of minimizing a Jensen-$f_α$-divergence, a natural generalization of the Jensen-Shannon divergence, where $f_α$ is a convex function expressed in terms of the loss function $\mathcal{L}_α$. It is also demonstrated that this $\mathcal{L}_α$-GAN problem recovers as special cases a number of GAN problems in the literature, including VanillaGAN, Least Squares GAN (LSGAN), Least $k$th order GAN (L$k$GAN) and the recently introduced $(α_D,α_G)$-GAN with $α_D=1$. Finally, experimental results are conducted on three datasets, MNIST, CIFAR-10, and Stacked MNIST to illustrate the performance of various examples of the $\mathcal{L}_α$-GAN system.

A Unifying Generator Loss Function for Generative Adversarial Networks

TL;DR

It is shown that this Lα-GAN problem recovers as special cases a number of GAN problems in the literature, including VanillaGAN, least squares GAN (LSGAN), least kth-order GAN (LkGAN), and the recently introduced (αD,αG)-GAN with αD=1.

Abstract

A unifying -parametrized generator loss function is introduced for a dual-objective generative adversarial network (GAN), which uses a canonical (or classical) discriminator loss function such as the one in the original GAN (VanillaGAN) system. The generator loss function is based on a symmetric class probability estimation type function, , and the resulting GAN system is termed -GAN. Under an optimal discriminator, it is shown that the generator's optimization problem consists of minimizing a Jensen--divergence, a natural generalization of the Jensen-Shannon divergence, where is a convex function expressed in terms of the loss function . It is also demonstrated that this -GAN problem recovers as special cases a number of GAN problems in the literature, including VanillaGAN, Least Squares GAN (LSGAN), Least th order GAN (LGAN) and the recently introduced -GAN with . Finally, experimental results are conducted on three datasets, MNIST, CIFAR-10, and Stacked MNIST to illustrate the performance of various examples of the -GAN system.
Paper Structure (15 sections, 8 theorems, 60 equations, 4 figures, 12 tables, 3 algorithms)

This paper contains 15 sections, 8 theorems, 60 equations, 4 figures, 12 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Main Results
  4. VanillaGAN
  5. $\alpha$-GAN
  6. Shifted LkGANs and LSGANs
  7. Experiments
  8. Experimental Setup
  9. Experimental Results
  10. Discussion
  11. Experiment 1
  12. Experiment 2
  13. Conclusion
  14. Neural Network Architectures
  15. Algorithms

Key Result

Lemma 1

Let $p$ and $q$ be two densities with common support $\mathcal{R} \subseteq \mathbb{R}^d$, and consider the function $f: [0, \infty) \to (-\infty,\infty]$ given by $f(u) = u\log u$. Then we have that

Figures (4)

  • Figure 1: Generated images for the best-performing ($\alpha_D$, $\alpha_G$)-GANs.
  • Figure 2: Average FID scores vs. epochs for various $(\alpha_D,\alpha_G)$-GANs.
  • Figure 3: Generated images for best-performing SL$k$GANs.
  • Figure 4: FID scores vs. epochs for various SL$k$GANs.

Theorems & Definitions (18)

  • Definition 1
  • Definition 2
  • Lemma 1
  • Remark 1: Jensen-$f$-divergence is a symmetric $f$-divergence
  • Remark 2: Domain of $f$
  • Definition 3
  • Theorem 1
  • Remark 3
  • Lemma 2
  • Definition 4
  • ...and 8 more