Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities

TL;DR

The paper introduces LS-GAN, a loss-function–based GAN with Lipschitz-density regularization to ensure the generated data matches the real data distribution and generalizes well. It extends the framework to GLS-GAN, unifying LS-GAN and WGAN, and to CLS-GAN for conditional and semi-supervised learning. The authors provide theoretical results on distributional consistency, PAC-style generalization bounds, and non-parametric analysis, along with gradient-penalty schemes to control Lipschitz constants. Empirically, LS-GAN, GLS-GAN, and CLS-GAN demonstrate competitive image generation and superior classification performance on standard benchmarks, with improved generalization as measured by MRE. Overall, the work offers a principled, regularized pathway for stable GAN training and versatile conditioned generation with solid theoretical guarantees.

Abstract

In this paper, we present the Lipschitz regularization theory and algorithms for a novel Loss-Sensitive Generative Adversarial Network (LS-GAN). Specifically, it trains a loss function to distinguish between real and fake samples by designated margins, while learning a generator alternately to produce realistic samples by minimizing their losses. The LS-GAN further regularizes its loss function with a Lipschitz regularity condition on the density of real data, yielding a regularized model that can better generalize to produce new data from a reasonable number of training examples than the classic GAN. We will further present a Generalized LS-GAN (GLS-GAN) and show it contains a large family of regularized GAN models, including both LS-GAN and Wasserstein GAN, as its special cases. Compared with the other GAN models, we will conduct experiments to show both LS-GAN and GLS-GAN exhibit competitive ability in generating new images in terms of the Minimum Reconstruction Error (MRE) assessed on a separate test set. We further extend the LS-GAN to a conditional form for supervised and semi-supervised learning problems, and demonstrate its outstanding performance on image classification tasks.

Figures (11)

• Figure 1: Comparison between two optimal loss functions $\widetilde{L}_{\theta^*}$ and $\widehat{L}_{\theta^*}$ in $\mathcal{F}_\kappa$ for LS-GAN. They are upper and lower bounds of the class of optimal loss functions $L_{\theta^*}$ to Problem (\ref{['eq:theta1']}). Both the upper and the lower bounds are cone-shaped, and have non-vanishing gradient almost everywhere. Specifically, in this one-dimensional example, both bounds are piecewise linear, having a slope of $\pm\kappa$ almost everywhere.
• Figure 2: Images generated by the LS-GAN on the CelebA dataset, in which the margin is computed as the distance between the features extracted from the Inception and VGG-16 networks. Images are resized to $128\times 128$ to fit the input size of both networks.
• Figure 3: Images generated by the DCGAN and the LS-GAN on the CelebA dataset. The results are obtained after $25$ epochs of training the models.
• Figure 4: Images generated by the DCGAN and the LS-GAN on the CelebA dataset without batch normalization for the generator networks. The results are obtained after $25$ epochs of training the models.
• Figure 5: The log of the generator's gradient norm over iterations. The generator is updated every 1, 3, and 5 iterations while the loss function is updated every iteration. The loss function can be quickly updated to be optimal, and the figure shows the generator's gradient does not vanish even if the loss function is well trained.

Theorems & Definitions (21)

• Lemma 1
• Lemma 2
• Remark 1
• Theorem 1
• Theorem 2
• Theorem 3
• Lemma 3
• Lemma 4
• Theorem 4
• Corollary 1