Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation

Shuting He, Peilin Ji, Yitong Yang, Changshuo Wang, Jiayi Ji, Yinglin Wang, Henghui Ding

TL;DR

The paper addresses the gap in understanding downstream tasks for 3D Gaussian Splatting (3DGS) beyond view synthesis by organizing segmentation, editing, and generation within a unified survey. It analyzes how 2D foundation models and NeRF-inspired methods inform 3DGS design, and categorizes techniques by architecture, supervision, and prompting style, supplemented with benchmark-driven performance insights. The work highlights current datasets, evaluation protocols, and performance trends, and articulates future directions such as large-scale feed-forward training, improved 3D-specific metrics, LLM integration, and generalist 3DGS models. Overall, this survey provides a comprehensive, actionable resource to accelerate research and development in high-level 3D understanding with 3DGS.

Abstract

3D Gaussian Splatting (3DGS) has recently emerged as a powerful alternative to Neural Radiance Fields (NeRF) for 3D scene representation, offering high-fidelity photorealistic rendering with real-time performance. Beyond novel view synthesis, the explicit and compact nature of 3DGS enables a wide range of downstream applications that require geometric and semantic understanding. This survey provides a comprehensive overview of recent progress in 3DGS applications. It first introduces 2D foundation models that support semantic understanding and control in 3DGS applications, followed by a review of NeRF-based methods that inform their 3DGS counterparts. We then categorize 3DGS applications into segmentation, editing, generation, and other functional tasks. For each, we summarize representative methods, supervision strategies, and learning paradigms, highlighting shared design principles and emerging trends. Commonly used datasets and evaluation protocols are also summarized, along with comparative analyses of recent methods across public benchmarks. To support ongoing research and development, a continually updated repository of papers, code, and resources is maintained at https://github.com/heshuting555/Awesome-3DGS-Applications.

A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation

TL;DR

The paper addresses the gap in understanding downstream tasks for 3D Gaussian Splatting (3DGS) beyond view synthesis by organizing segmentation, editing, and generation within a unified survey. It analyzes how 2D foundation models and NeRF-inspired methods inform 3DGS design, and categorizes techniques by architecture, supervision, and prompting style, supplemented with benchmark-driven performance insights. The work highlights current datasets, evaluation protocols, and performance trends, and articulates future directions such as large-scale feed-forward training, improved 3D-specific metrics, LLM integration, and generalist 3DGS models. Overall, this survey provides a comprehensive, actionable resource to accelerate research and development in high-level 3D understanding with 3DGS.

Abstract

3D Gaussian Splatting (3DGS) has recently emerged as a powerful alternative to Neural Radiance Fields (NeRF) for 3D scene representation, offering high-fidelity photorealistic rendering with real-time performance. Beyond novel view synthesis, the explicit and compact nature of 3DGS enables a wide range of downstream applications that require geometric and semantic understanding. This survey provides a comprehensive overview of recent progress in 3DGS applications. It first introduces 2D foundation models that support semantic understanding and control in 3DGS applications, followed by a review of NeRF-based methods that inform their 3DGS counterparts. We then categorize 3DGS applications into segmentation, editing, generation, and other functional tasks. For each, we summarize representative methods, supervision strategies, and learning paradigms, highlighting shared design principles and emerging trends. Commonly used datasets and evaluation protocols are also summarized, along with comparative analyses of recent methods across public benchmarks. To support ongoing research and development, a continually updated repository of papers, code, and resources is maintained at https://github.com/heshuting555/Awesome-3DGS-Applications.

Paper Structure

This paper contains 30 sections, 2 equations, 3 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Problem Formulation and Taxonomy
  4. Application Tasks Category
  5. Learning Paradigms for Application Tasks
  6. 2D Foundation Model
  7. Related Research Areas
  8. 3DGS Application Tasks
  9. 3DGS Segmentation
  10. Feature Distillation-based Methods
  11. 2D Mask Lifting-based Methods
  12. Feed-Forward-based Methods
  13. 3DGS Editing
  14. Editing Based on Text Prompt
  15. Editing Based on Image Prompt
  16. ...and 15 more sections

Figures (3)

  • Figure 1: Overview of the three main 3DGS applications. (a) Segmentation: Segment each semantic region. (b) Editing: Perform editing via text or image prompt. (c) Generation: Support text-to-3D and image-to-3D generation.
  • Figure 2: Overview of the structure of this survey.
  • Figure 3: Examples from 13 commonly used datasets for segmentation, editing, and generation.