3D Gaussian as a New Era: A Survey

TL;DR

The paper surveys recent advances in 3D Gaussian Splatting (3D-GS), an explicit radiance-field representation that uses Gaussians for efficient rendering and reconstruction without neural radiance fields. It presents a unified framework and taxonomy spanning representation, reconstruction, manipulation, perception, generation, and virtual humans, and highlights memory compression, photorealism, dynamic 4D extensions, diffusion-guided generation, and perception-enabled applications. Key contributions include memory-efficient scaffolding (e.g., Scaffold-GS, VQ compression), photorealism improvements (multi-scale Gaussians, lighting decomposition), few-shot and monocular reconstruction, interactive editing tools, diffusion-based object- and scene-level generation, and extensive virtual-human pipelines. The survey also discusses open challenges—floats, occlusion, normal estimation, large-scale dynamic scenes, and 4D consistency—and outlines future directions such as integrating with foundation models, physics-based rendering, and scalable 4D generation for practical deployment.

Abstract

3D Gaussian Splatting (3D-GS) has emerged as a significant advancement in the field of Computer Graphics, offering explicit scene representation and novel view synthesis without the reliance on neural networks, such as Neural Radiance Fields (NeRF). This technique has found diverse applications in areas such as robotics, urban mapping, autonomous navigation, and virtual reality/augmented reality, just name a few. Given the growing popularity and expanding research in 3D Gaussian Splatting, this paper presents a comprehensive survey of relevant papers from the past year. We organize the survey into taxonomies based on characteristics and applications, providing an introduction to the theoretical underpinnings of 3D Gaussian Splatting. Our goal through this survey is to acquaint new researchers with 3D Gaussian Splatting, serve as a valuable reference for seminal works in the field, and inspire future research directions, as discussed in our concluding section.

Figures (8)

• Figure 1: (Top) The structure of this survey. We begin by introducing the optimization of 3D-GS in terms of efficiency, photorealism, costs, and physics. Then, we review 3D-GS on reconstruction, manipulation, perception, generation, and virtual human applications. (Bottom) The monthly number of publications since July 2023.
• Figure 2: Taxonomy of existing 3D Gaussian Splatting derived methods.
• Figure 3: An illustration of refining the representation of 3D-GS: (a) efficiencies, (b) photorealism, (c) costs, and (d) physics. Images courtesy of jiang2023gaussianshaderkratimenos2023dynmfzhu2023fsgs.
• Figure 4: An illustration of 3D-GS on reconstruction: (a) static reconstruction, (b) dynamic reconstruction. Images courtesy of guedon2023sugarjiang20233d.
• Figure 5: An illustration of employing sparse control points and a deformation MLP to direct 3D Gaussian dynamics. Images courtesy of huang2023sc.