Asymmetric regularization mechanism for GAN training with Variational Inequalities

TL;DR

This work recasts GAN training as a variational-inequality problem and introduces an asymmetric, discriminator-focused regularization comprising Tikhonov and zero-centered gradient penalties to inject curvature without shifting the equilibrium. Under smoothness and a local Gauss–Newton identifiability condition, it derives explicit Lipschitz and strong monotonicity constants for the regularized operator, enabling provable last-iterate linear convergence of Extrapolation-from-the-Past (EFTP). The analysis demonstrates that discriminator-side curvature improves conditioning and stabilizes dynamics, even when strong monotonicity does not hold, with numerical evidence on a toy example. The proposed framework provides a principled approach to stabilize GAN training via VI theory and explicit convergence guarantees, and it lays groundwork for extensions to stochastic settings and multi-agent games.

Abstract

We formulate the training of generative adversarial networks (GANs) as a Nash equilibrium seeking problem. To stabilize the training process and find a Nash equilibrium, we propose an asymmetric regularization mechanism based on the classic Tikhonov step and on a novel zero-centered gradient penalty. Under smoothness and a local identifiability condition induced by a Gauss-Newton Gramian, we obtain explicit Lipschitz and (strong)-monotonicity constants for the regularized operator. These constants ensure last-iterate linear convergence of a single-call Extrapolation-from-the-Past (EFTP) method. Empirical simulations on an academic example show that, even when strong monotonicity cannot be achieved, the asymmetric regularization is enough to converge to an equilibrium and stabilize the trajectory.

Figures (3)

• Figure 1: Trajectories with discriminator-only curvature: (a) FB diverges for $\gamma=0$; (b) FB converges for $\gamma=2$; (c) EG ($\gamma=2$); (d) EFTP ($\gamma=2$). Blue dots indicate the starting point and squares the end points; the star is the solution. The legends report iteration till convergence.
• Figure 2: Residual vs. iterations on the bilinear toy for $\gamma=\{0,0.5,2\}$.
• Figure 3: Residual as a function of gradient evaluations.

Theorems & Definitions (18)

• Remark 1
• Remark 2
• Remark 3
• Remark 4
• Lemma 1: PSD and uniform bound for the Gramian
• proof
• Proposition 1: Lipschitz constant of $F_\gamma$
• proof
• Remark 5
• Proposition 2: Strong monotonicity of $F_\gamma$