GauStudio: A Modular Framework for 3D Gaussian Splatting and Beyond

TL;DR

GauStudio introduces a modular framework for 3D Gaussian Splatting that unifies foreground and background representations with a plug-and-play pipeline for initialization, optimization, enhancement, and compression. It adds GauS, a surface reconstruction module based on volumetric fusion, to convert Gaussians into textured meshes efficiently and at scale. A dedicated Gaussian Sky Modeling component separates sky from foreground using semantic priors to reduce artifacts in unbounded outdoor scenes. Together, GauStudio and GauS improve novel view synthesis and mesh reconstruction while enabling flexible, scalable 3D scene modeling across reconstruction, editing, and simulation tasks.

Abstract

We present GauStudio, a novel modular framework for modeling 3D Gaussian Splatting (3DGS) to provide standardized, plug-and-play components for users to easily customize and implement a 3DGS pipeline. Supported by our framework, we propose a hybrid Gaussian representation with foreground and skyball background models. Experiments demonstrate this representation reduces artifacts in unbounded outdoor scenes and improves novel view synthesis. Finally, we propose Gaussian Splatting Surface Reconstruction (GauS), a novel render-then-fuse approach for high-fidelity mesh reconstruction from 3DGS inputs without fine-tuning. Overall, our GauStudio framework, hybrid representation, and GauS approach enhance 3DGS modeling and rendering capabilities, enabling higher-quality novel view synthesis and surface reconstruction.

Figures (7)

• Figure 1: GauStudio is a modular framework that unifies various 3D Gaussian Splatting techniques. It decomposes 3D scenes into components like foreground and background models, represented using specialized techniques on 3D Gaussians. These components can be flexibly combined and rendered to synthesize novel views, enabling tailored modeling pipelines for different tasks.
• Figure 2: Comparsion with other mesh extraction strategies. Utilizing the Poisson reconstruction directly on 3DGS leads to a bad performance. Due to the special design of GaussianPro on normal and depth, the performance of Poisson reconstruction on GaussianPro is slightly better. SuGaR always tends to produce larger ellipses. Compared with the above methods, our method can generate high-quality mesh efficiently.
• Figure 3: Our Gaussian Surface Reconstrution can be integrated into existing 3DGS pipelines. Before GS rasterization, 3DGS-based frameworks will convert Gaussian Splatting into a unified representation that can be used for our Gaussian Surface Reconstruction.
• Figure 4: Qualitative comparisons of Surface Reconstruciton on Blender Dataset
• Figure 5: Qualitative comparisons between SuGaR and GauS on BlendedMVS Dataset