Recent Advances in 3D Gaussian Splatting

TL;DR

This survey analyzes Recent Advances in 3D Gaussian Splatting (3DGS), an explicit and fast 3D representation built from Gaussian ellipsoids that enables rasterization-based rendering and straightforward editing. It covers reconstruction, editing, and application domains, highlighting techniques for quality enhancement, compression, dynamic 3D/4D extensions, and strategies for handling challenging inputs. The paper also surveys downstream uses such as segmentation, SLAM, digital humans, and diffusion-driven 3D/4D generation, comparing 3DGS to NeRF and other representations. It concludes with a candid discussion of current limitations and future directions, including robustness, geometry quality, independent editing, 4D realism, and platform compatibility, suggesting hybrid approaches and open-source tooling to accelerate progress. 3DGS stands out for its real-time rendering, editable geometry, and compatibility with diffusion priors for 3D content creation, with broad implications for virtual production, robotics, and interactive media.

Abstract

The emergence of 3D Gaussian Splatting (3DGS) has greatly accelerated the rendering speed of novel view synthesis. Unlike neural implicit representations like Neural Radiance Fields (NeRF) that represent a 3D scene with position and viewpoint-conditioned neural networks, 3D Gaussian Splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images. Apart from the fast rendering speed, the explicit representation of 3D Gaussian Splatting facilitates editing tasks like dynamic reconstruction, geometry editing, and physical simulation. Considering the rapid change and growing number of works in this field, we present a literature review of recent 3D Gaussian Splatting methods, which can be roughly classified into 3D reconstruction, 3D editing, and other downstream applications by functionality. Traditional point-based rendering methods and the rendering formulation of 3D Gaussian Splatting are also illustrated for a better understanding of this technique. This survey aims to help beginners get into this field quickly and provide experienced researchers with a comprehensive overview, which can stimulate the future development of the 3D Gaussian Splatting representation.

Figures (14)

• Figure 1: Structure of the literature review and taxonomy of current 3D Gaussian Splatting methods.
• Figure 2: A brief timeline of representative works with the 3D Gaussian Splatting representation.
• Figure 3: Overview of GaussianPro GaussianPro. Neighboring views' normal direction consistency is considered to produce better reconstruction results.
• Figure 4: Pipeline from EAGLES girish2023eagles. Vector Quantization (VQ) is utilized to compress Gaussian attributes.
• Figure 5: The results of Deformable3DGS yang2023deformable. Given a set of monocular multi-view images (a), this method can achieve novel view synthesis (b) and time synthesis (c), and has better rendering quality compared to HyperNeRF park2021hypernerf (d).