A Novel Generator with Auxiliary Branch for Improving GAN Performance

TL;DR

This work tackles GAN training instability and information-flow bottlenecks in residual-block generators by introducing a two-branch generator with an auxiliary branch and a gated feature fusion module (GFFM) to propagate coarse information to later layers. The AB block uses a gating mechanism to blend main and auxiliary features, improving information flow with controllable fusion. Empirical results across CIFAR-10/100, LSUN, CelebA-HQ, AFHQ, and tiny-ImageNet show significant improvements in FID and IS, with ablations validating the contribution of the auxiliary branch and GFFM. The approach achieves substantial gains with only modest increases in parameters and demonstrates generalization to high-resolution and conditional GAN settings, suggesting broad applicability in GAN-based image synthesis.

Abstract

The generator in the generative adversarial network (GAN) learns image generation in a coarse-to-fine manner in which earlier layers learn the overall structure of the image and the latter ones refine the details. To propagate the coarse information well, recent works usually build their generators by stacking up multiple residual blocks. Although the residual block can produce a high-quality image as well as be trained stably, it often impedes the information flow in the network. To alleviate this problem, this brief introduces a novel generator architecture that produces the image by combining features obtained through two different branches: the main and auxiliary branches. The goal of the main branch is to produce the image by passing through the multiple residual blocks, whereas the auxiliary branch is to convey the coarse information in the earlier layer to the later one. To combine the features in the main and auxiliary branches successfully, we also propose a gated feature fusion module that controls the information flow in those branches. To prove the superiority of the proposed method, this brief provides extensive experiments using various standard datasets including CIFAR-10, CIFAR-100, LSUN, CelebA-HQ, AFHQ, and tiny-ImageNet. Furthermore, we conducted various ablation studies to demonstrate the generalization ability of the proposed method. Quantitative evaluations prove that the proposed method exhibits impressive GAN performance in terms of Inception score (IS) and Frechet inception distance (FID). For instance, the proposed method boosts the FID and IS scores on the tiny-ImageNet dataset from 35.13 to 25.00 and 20.23 to 25.57, respectively.

Figures (7)

• Figure 1: The overall generator architectures and residual blocks utilized in SNGAN miyato2018spectralmiyato2018cgans and BigGAN brock2018large. (a) Generator architecture for SNGAN, (b) Generator architecture for BigGAN, (c) Residual block for SNGAN, (d) Residual block for BigGAN.
• Figure 2: The overall generator architecture of the proposed method. The proposed method builds the generator using multiple AB blocks which contain the main and auxiliary branches.
• Figure 3: Detailed architecture of the proposed GFFM. BN and Conv are the batch normalization ioffe2015batch and convolution operation, respectively. In addition, $\sigma(\cdot)$ indicates the sigmoid function.
• Figure 4: Comparison of FID scores over training iterations. Blue, red, and yellow lines indicate the FID scores of the proposed method, BigGAN, and SNGAN, respectively.
• Figure 5: Comparison of IS scores over training iterations. Blue, red, and yellow lines indicate the IS scores of the proposed method, BigGAN, and SNGAN, respectively.