Rethinking Image Skip Connections in StyleGAN2

TL;DR

The paper analyzes image skip connections in StyleGAN2 and identifies a theoretical limitation when high-dimensional feature concatenations are projected to RGB. It introduces the image squeeze connection, which compresses channels to $c/r$, uses a squeeze path to compose a low-dimensional toRGB feature, and then excites back to the original dimension before fusion, implemented via a final $1\times1$ convolution. Across CIFAR-10, FFHQ, LSUN, and AFHQ, it yields consistent improvements in FID, IS, and precision/recall while reducing parameters by about $12.1\%$, demonstrating practical gains without substantial architectural complexity. The approach is simple to integrate with existing StyleGAN2-based models and points to a promising direction for efficient, high-quality image synthesis.

Abstract

Various models based on StyleGAN have gained significant traction in the field of image synthesis, attributed to their robust training stability and superior performances. Within the StyleGAN framework, the adoption of image skip connection is favored over the traditional residual connection. However, this preference is just based on empirical observations; there has not been any in-depth mathematical analysis on it yet. To rectify this situation, this brief aims to elucidate the mathematical meaning of the image skip connection and introduce a groundbreaking methodology, termed the image squeeze connection, which significantly improves the quality of image synthesis. Specifically, we analyze the image skip connection technique to reveal its problem and introduce the proposed method which not only effectively boosts the GAN performance but also reduces the required number of network parameters. Extensive experiments on various datasets demonstrate that the proposed method consistently enhances the performance of state-of-the-art models based on StyleGAN. We believe that our findings represent a vital advancement in the field of image synthesis, suggesting a novel direction for future research and applications.

Figures (6)

• Figure 1: The overall architecture of the G in StyleGAN2, which utilizes the image skip connections. The mapping network, which generates the style vector, is excluded for clarity in the illustration.
• Figure 2: The overall architecture of the proposed method.
• Figure 3: Sample images of the proposed method on the CIFAR-10 dataset.
• Figure 4: Selective samples generated by our method. For FFHQ and LSUN datasets, we show the results of GGDR with our method.
• Figure 5: Selective samples generated by our method. For AFHQ datasets, we show the results of ADA with our method.