3D Gaussian Splatting in Robotics: A Survey

TL;DR

3D Gaussian Splatting (3DGS) provides an explicit, Gaussian-primitive radiance field enabling real-time rendering and differentiable optimization, addressing NeRF's inefficiencies for robotics. The survey organizes robotics-oriented work into scene understanding, interaction, and algorithmic advances that improve adaptability and efficiency, and it aggregates datasets and evaluation metrics across reconstruction, SLAM, localization, segmentation, manipulation, and navigation. It also discusses challenges in dynamic scenes, large-scale mapping, and sim-to-real transfer, proposing robust tracking, lifelong mapping, large-scale relocalization, and diffusion-guided editing as key future directions. Overall, the paper highlights 3DGS as a versatile tool for dense, editable scene representations in robotics with strong potential for real-world deployment.

Abstract

Dense 3D representations of the environment have been a long-term goal in the robotics field. While previous Neural Radiance Fields (NeRF) representation have been prevalent for its implicit, coordinate-based model, the recent emergence of 3D Gaussian Splatting (3DGS) has demonstrated remarkable potential in its explicit radiance field representation. By leveraging 3D Gaussian primitives for explicit scene representation and enabling differentiable rendering, 3DGS has shown significant advantages over other radiance fields in real-time rendering and photo-realistic performance, which is beneficial for robotic applications. In this survey, we provide a comprehensive understanding of 3DGS in the field of robotics. We divide our discussion of the related works into two main categories: the application of 3DGS and the advancements in 3DGS techniques. In the application section, we explore how 3DGS has been utilized in various robotics tasks from scene understanding and interaction perspectives. The advance of 3DGS section focuses on the improvements of 3DGS own properties in its adaptability and efficiency, aiming to enhance its performance in robotics. We then summarize the most commonly used datasets and evaluation metrics in robotics. Finally, we identify the challenges and limitations of current 3DGS methods and discuss the future development of 3DGS in robotics.

Figures (15)

• Figure 1: A taxonomy of 3D Gaussian Splatting (3DGS) in Robotics.
• Figure 2: An illustration of the forward process of 3DGS.
• Figure 3: Chronological: 3DGS for Scene Reconstruction. The red dots in the figure represent months, which also applys to the other figures.
• Figure 4: An illustration of 3DGS for dynamic reconstruction. Fig. \ref{['fig:dynamic_time_varying']}, Fig. \ref{['fig:dynamic_deformation']}, and Fig. \ref{['fig:dynamic_4dgs']} are originally shown in li2024spacetime, yang2024deformable, and yang2023real, respectively.
• Figure 5: Chronological: 3DGS for Scene Segmentation and Editing.