Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dropout-GAN: Learning from a Dynamic Ensemble of Discriminators

Gonçalo Mordido, Haojin Yang, Christoph Meinel

TL;DR

Dropout-GAN tackles mode collapse by training a single generator against a dynamic ensemble of discriminators, achieved by dropping discriminator feedback with probability $d$ at the end of each batch. This adversarial dropout regularizes learning and forces G to cover multiple data modes, improving sample diversity and training stability as evidenced by FID improvements across MNIST, CIFAR-10, and CelebA and by favorable comparisons with UnrolledGAN, D2GAN, and MGAN. The method is flexible, enhances several GAN variants, and scales with more discriminators, albeit with longer training. Future directions include adapting learning rates to the ensemble size and introducing game-theoretic interactions among discriminators to further exploit multi-adversarial training benefits.

Abstract

We propose to incorporate adversarial dropout in generative multi-adversarial networks, by omitting or dropping out, the feedback of each discriminator in the framework with some probability at the end of each batch. Our approach forces the single generator not to constrain its output to satisfy a single discriminator, but, instead, to satisfy a dynamic ensemble of discriminators. We show that this leads to a more generalized generator, promoting variety in the generated samples and avoiding the common mode collapse problem commonly experienced with generative adversarial networks (GANs). We further provide evidence that the proposed framework, named Dropout-GAN, promotes sample diversity both within and across epochs, eliminating mode collapse and stabilizing training.

Dropout-GAN: Learning from a Dynamic Ensemble of Discriminators

TL;DR

Dropout-GAN tackles mode collapse by training a single generator against a dynamic ensemble of discriminators, achieved by dropping discriminator feedback with probability at the end of each batch. This adversarial dropout regularizes learning and forces G to cover multiple data modes, improving sample diversity and training stability as evidenced by FID improvements across MNIST, CIFAR-10, and CelebA and by favorable comparisons with UnrolledGAN, D2GAN, and MGAN. The method is flexible, enhances several GAN variants, and scales with more discriminators, albeit with longer training. Future directions include adapting learning rates to the ensemble size and introducing game-theoretic interactions among discriminators to further exploit multi-adversarial training benefits.

Abstract

We propose to incorporate adversarial dropout in generative multi-adversarial networks, by omitting or dropping out, the feedback of each discriminator in the framework with some probability at the end of each batch. Our approach forces the single generator not to constrain its output to satisfy a single discriminator, but, instead, to satisfy a dynamic ensemble of discriminators. We show that this leads to a more generalized generator, promoting variety in the generated samples and avoiding the common mode collapse problem commonly experienced with generative adversarial networks (GANs). We further provide evidence that the proposed framework, named Dropout-GAN, promotes sample diversity both within and across epochs, eliminating mode collapse and stabilizing training.

Paper Structure

This paper contains 16 sections, 3 equations, 8 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Generative Adversarial Networks
  3. Dropout-GAN
  4. Implementation Details
  5. Number of Discriminators
  6. Batch Partitioning
  7. Dropout Rate
  8. Experimental Results
  9. MNIST
  10. CIFAR-10
  11. CelebA
  12. Parameter Evaluation
  13. Fréchet Inception Distance
  14. Method evaluation
  15. Related Work
  16. ...and 1 more sections

Figures (8)

  • Figure 1: We expand the original GAN framework (left) to multiple adversaries, where some discriminators are dropped out according to some probability (right), leading to only a random subset of feedback (represented by the arrows) being used by $G$ at the end of each batch.
  • Figure 2: MNIST results using different combinations of the number of discriminators and dropout rates.
  • Figure 3: CIFAR-10 results using different combinations of the number of discriminators and dropout rates.
  • Figure 4: CelebA results using different combinations of the number of discriminators and dropout rates.
  • Figure 5: Mean FID calculated across 40 epochs on the different datasets. Smaller values mean better looking and more varied generated samples over time. The convex representation of FID indicates the benefits in using mid-range dropout rates.
  • ...and 3 more figures