Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step

TL;DR

The paper challenges the view that GAN training must monotonically decrease a divergence at every step. Through synthetic and real-data experiments across NS-GAN, WGAN, and gradient-penalty variants, it shows that learning can reach Nash equilibria via trajectories that do not strictly minimize a divergence, and that gradient penalties improve stability and robustness even when their theoretical divergence interpretation is unclear. It demonstrates that NS-GANs can fit data on low-dimensional manifolds where JS-divergence provides little gradient, and that gradient penalties (GAN-GP, DRAGAN-NS) enhance convergence and sample quality across tasks. The findings suggest a broader, dynamics-focused perspective on GAN optimization with practical implications for model design and training protocols.

Abstract

Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion. Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost. GANs are designed to reach a Nash equilibrium at which each player cannot reduce their cost without changing the other players' parameters. One useful approach for the theory of GANs is to show that a divergence between the training distribution and the model distribution obtains its minimum value at equilibrium. Several recent research directions have been motivated by the idea that this divergence is the primary guide for the learning process and that every step of learning should decrease the divergence. We show that this view is overly restrictive. During GAN training, the discriminator provides learning signal in situations where the gradients of the divergences between distributions would not be useful. We provide empirical counterexamples to the view of GAN training as divergence minimization. Specifically, we demonstrate that GANs are able to learn distributions in situations where the divergence minimization point of view predicts they would fail. We also show that gradient penalties motivated from the divergence minimization perspective are equally helpful when applied in other contexts in which the divergence minimization perspective does not predict they would be helpful. This contributes to a growing body of evidence that GAN training may be more usefully viewed as approaching Nash equilibria via trajectories that do not necessarily minimize a specific divergence at each step.

Figures (22)

• Figure 1: Visualization of experiment 1 training dynamics in two dimensions, demonstrated specifically in the case where the model is initialized so that it it represents a linear manifold parallel to the linear manifold of the training data. Here the GAN model (red points) converges upon the one dimensional synthetic data distribution (blue points). Specifically, this is an illustration of the parallel line thought experiment from wgan. When run in practice with a non-saturating GAN, the GAN succeeds. In the same setting, minimization of Jensen-Shannon divergence would fail. This indicates that while Jensen-Shannon divergence is useful for characterizing GAN equilibrium, it does not necessarily tell us much about non-equilibrium learning dynamics.
• Figure 2: (Left) A recreation of Figure 2 of wgan. This figure is used by wgan to show that a model they call the "traditional GAN" suffers from vanishing gradients in the areas where $D(x)$ is flat. This plot is correct if "traditional GAN" is used to refer to the minimax GAN, but it does not apply to the non-saturating GAN. (Right) A plot of both generator losses from the original GAN paper, as a function of the generator output. Even when the model distribution is highly separated from the data distribution, non-saturating GANs are able to bring the model distribution closer to the data distribution because the loss function has strong gradient when the generator samples are far from the data samples, even when the discriminator itself has nearly zero gradient. While it is true that the $\frac{1}{2} \log( 1 - D(x))$ loss has a vanishing gradient on the right half of the plot, the original GAN paper instead recommends implementing $-\frac{1}{2} \log D(x)$. This latter, recommended loss function has a vanishing gradient only on the left side of the plot. It makes sense for the gradient to vanish on the left because generator samples in that area have already reached the area where data samples lie.
• Figure 3: Visualization of experiment 1 training dynamics in two dimensions. Here the GAN model (red points) converges upon the one dimensional synthetic data distribution (blue points). We note that this is a visual illustration, and the results have not been averaged out over multiple seeds. Exact plots may vary on different runs. However, a single example of success is sufficient to refute claims that this this task is impossible for this model.
• Figure 4: Visualization of experiment 2 training dynamics in two dimensions - where the GAN model has 3 latent variables. Here the rank one GAN model (red points) converges upon the one dimensional synthetic data distribution (blue points). We observe how for poor initialization the non-saturating GAN suffers from mode collapse. However, adding a gradient penalty stabilizes training. We note that this is a visual illustration, and the results have not been averaged out over multiple seeds. Exact plots may vary on different runs.
• Figure 5: The square Fréchet distance between the learned Gaussian and the true Gaussian distribution. For reference, we also plot the distance obtained by a randomly initialized generator with the same architecture as the trained generators. Results are averaged over 1000 runs. Lower values are better.