pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
Eric R. Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, Gordon Wetzstein
TL;DR
pi-GAN addresses the challenge of 3D-aware image synthesis by learning a view-consistent radiance field with a sinusoidally activated network conditioned via a FiLM-mapped mapping network. By applying differentiable volume rendering and a progressively growing discriminator, it delivers high-detail, view-consistent images and yields interpretable 3D proxy geometry. The approach achieves state-of-the-art results on real and synthetic datasets, demonstrates extrapolation to unseen viewpoints, and enables single-view reconstruction via inverse rendering. Overall, pi-GAN advances unconditional 3D-aware generation and provides a framework for efficient future refinements in implicit neural representations and rendering.
Abstract
We have witnessed rapid progress on 3D-aware image synthesis, leveraging recent advances in generative visual models and neural rendering. Existing approaches however fall short in two ways: first, they may lack an underlying 3D representation or rely on view-inconsistent rendering, hence synthesizing images that are not multi-view consistent; second, they often depend upon representation network architectures that are not expressive enough, and their results thus lack in image quality. We propose a novel generative model, named Periodic Implicit Generative Adversarial Networks ($π$-GAN or pi-GAN), for high-quality 3D-aware image synthesis. $π$-GAN leverages neural representations with periodic activation functions and volumetric rendering to represent scenes as view-consistent 3D representations with fine detail. The proposed approach obtains state-of-the-art results for 3D-aware image synthesis with multiple real and synthetic datasets.