IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks

TL;DR

IB-GAN advances unsupervised disentangled representation learning by infusing the Information Bottleneck framework into a GAN, introducing an intermediate stochastic latent layer that enables a trainable, factorized latent distribution. The model optimizes a GAN objective together with variational bounds on the generative mutual information, using a latent encoder and a prior to control information flow from latent codes to generated data. Empirically, IB-GAN achieves competitive disentanglement scores on synthetic datasets and delivers high-quality, diverse samples on CelebA and 3D Chairs, outperforming InfoGAN and approaching or matching $eta$-VAE variants. The approach offers a principled, information-theoretic mechanism to promote disentanglement in GAN-based generative modeling, with practical impact for interpretable generative representations and robust downstream tasks.

Abstract

We propose a new GAN-based unsupervised model for disentangled representation learning. The new model is discovered in an attempt to utilize the Information Bottleneck (IB) framework to the optimization of GAN, thereby named IB-GAN. The architecture of IB-GAN is partially similar to that of InfoGAN but has a critical difference; an intermediate layer of the generator is leveraged to constrain the mutual information between the input and the generated output. The intermediate stochastic layer can serve as a learnable latent distribution that is trained with the generator jointly in an end-to-end fashion. As a result, the generator of IB-GAN can harness the latent space in a disentangled and interpretable manner. With the experiments on dSprites and Color-dSprites dataset, we demonstrate that IB-GAN achieves competitive disentanglement scores to those of state-of-the-art \b{eta}-VAEs and outperforms InfoGAN. Moreover, the visual quality and the diversity of samples generated by IB-GAN are often better than those by \b{eta}-VAEs and Info-GAN in terms of FID score on CelebA and 3D Chairs dataset.

Figures (12)

• Figure 1: Latent generative factors captured by IB-GAN on (a) dSprites and (b) Color-dSprites dataset. From top to bottom, each row corresponds to the factors of color, shape, scale, rotation, position Y, and position X.
• Figure 2: An illustration of IB-GAN. It adopts a representation encoder $e_\psi(r|z)$ and a KL-divergence loss derived from IB theory. Since the encoder $q_\phi(r|z)$ is assumed as Gaussian, it is convenient to define $m(r)$ as Gaussian. The MSE loss is computed by the decoder $\log q_\phi(z|x)$ in Eq.\ref{['eq:lower_ibgan']}.
• Figure 3: Effects of $\beta$ on the converged upper/lower-bound of MI and disentanglement metric scores Kim:2018th.
• Figure 4: The plot of variational upper-bound and lower-bound of MI with independent $\hbox{KL}(e(r_i|z)||m(r_i))$ values for all $r_i$ ($i=1,\ldots,10$) and disentanglement scores Kim:2018th over 150K training iterations. The vertical dashed black line represents the iteration at the highest disentanglement score.
• Figure 5: Latent traversals of IB-GAN that captures the factors of (a) azimuth, gender and skin tone attributes on CelebA and (b) scale, leg and azimuth on 3D Chairs. More factors captured by IB-GAN are presented in Appendix.