MCL-GAN: Generative Adversarial Networks with Multiple Specialized Discriminators

TL;DR

GAN mode collapse is mitigated by MCL-GAN, which employs multiple specialized discriminators as experts under a single generator. The framework combines expert and non-expert discriminator training, a balanced assignment mechanism via a KL-based loss, and an adaptive sparsity term to determine the active number of discriminators, all while sharing early-layer features to keep overhead low. Empirical results across unconditional and conditional generation tasks show improved mode coverage and sample fidelity, with clear evidence of semantic discriminator specialization and strong performance gains on both DCGAN and StyleGAN2 backbones. The approach offers a scalable, flexible alternative to prior multi-discriminator/generator designs, delivering robust diversity and quality with minimal additional cost.

Abstract

We propose a framework of generative adversarial networks with multiple discriminators, which collaborate to represent a real dataset more effectively. Our approach facilitates learning a generator consistent with the underlying data distribution based on real images and thus mitigates the chronic mode collapse problem. From the inspiration of multiple choice learning, we guide each discriminator to have expertise in a subset of the entire data and allow the generator to find reasonable correspondences between the latent and real data spaces automatically without extra supervision for training examples. Despite the use of multiple discriminators, the backbone networks are shared across the discriminators and the increase in training cost is marginal. We demonstrate the effectiveness of our algorithm using multiple evaluation metrics in the standard datasets for diverse tasks.

Figures (14)

• Figure 1: Snapshots of 256 random samples drawn from the generators of the baseline and MCL-GAN with (left) the standard GAN loss and (right) the Hinge loss after 1K, 5K, 10K, 20K and 50K steps. Data sampled from the true distribution are in orange while the generated ones are in green.
• Figure 2: Effect of the $\ell_1$ loss weight in MCL-GAN ($\gamma$). The graphs show the ratio of training examples associated with each discriminator.
• Figure 3: Generated image clusters by MCL-GAN. Each row represents the cluster associated with each discriminator ($M = 10$). Note that the images in the same row often have similar shapes and semantics but are not necessarily in the same class.
• Figure 4: The cosine similarity matrix among the attribute representations obtained by the discriminator assignments of MCL-GAN on the CelebA dataset.
• Figure 5: Visualization of Mapping between examples and discriminators with different $\ell_1$ loss weights ($\gamma$). Each sample is colored by its expert discriminator, and real data are in orange. The bar graphs show the ratio of training examples associated with each discriminator which are identical to those in Figure \ref{['fig:toy_l1']} of the main paper.