Distribution Fitting for Combating Mode Collapse in Generative Adversarial Networks

TL;DR

Mode collapse in GANs can arise from nonuniform sampling, where training batches fail to reflect the full data distribution. The authors propose Global Distribution Fitting (GDF) and Local Distribution Fitting (LDF), which add mean- and standard deviation–matching penalties to constrain the generated distribution while preserving the global GAN minimum. They provide theoretical arguments that these penalties preserve the minimum only when $p_g=p_{data}$ and demonstrate through experiments on Gaussian mixtures, stacked MNIST, and natural image datasets that GDF/LDF improve mode coverage and training stability with minimal computational overhead. The work offers a practical, lightweight approach to mitigating mode collapse by aligning distribution statistics during training.

Abstract

Mode collapse is a significant unsolved issue of generative adversarial networks. In this work, we examine the causes of mode collapse from a novel perspective. Due to the nonuniform sampling in the training process, some sub-distributions may be missed when sampling data. As a result, even when the generated distribution differs from the real one, the GAN objective can still achieve the minimum. To address the issue, we propose a global distribution fitting (GDF) method with a penalty term to confine the generated data distribution. When the generated distribution differs from the real one, GDF will make the objective harder to reach the minimal value, while the original global minimum is not changed. To deal with the circumstance when the overall real data is unreachable, we also propose a local distribution fitting (LDF) method. Experiments on several benchmarks demonstrate the effectiveness and competitive performance of GDF and LDF.

Figures (11)

• Figure 1: The dynamics of the model through time. The image rows from top to bottom represent the model without distribution fitting, the model with LDF, and the model with GDF, respectively.
• Figure 2: The dynamics of the model through time in an extreme case. The image rows from top to bottom represent the model without distribution fitting, the model with LDF, and the model with GDF, respectively.
• Figure 3: The convergence curves of (a) GDF and (b) LDF. The horizontal coordinate axis represents the training iteration and the vertical one represents the values.
• Figure 4: The samples in a mini-batch generated by GAN on MNIST, and the number of samples per class in the mini-batch.
• Figure 5: The samples in a mini-batch generated by GAN with GDF on MNIST, and the number of samples per class in the mini-batch.

Theorems & Definitions (17)

• Definition 1
• Lemma 1
• proof
• Theorem 1
• proof
• Theorem 2
• proof
• Corollary 1
• proof
• Lemma 2