NegGS: Negative Gaussian Splatting

TL;DR

The paper tackles dynamic 3D scene editing by extending Gaussian Splatting with Dynamic Multi-Gaussian Soup (D-MiSo), a two-tier Gaussian framework tied to Triangle Soup. Core-Gaussians capture global motion while Sub-Gaussians provide rendering detail, with time evolution governed by two deformation networks and a two-stage training regime. The approach enables intuitive editing at specific time steps, including mesh-based edits, direct Sub-Triangle Soup manipulation, and space-transform edits, while maintaining competitive reconstruction performance across multiple dynamic datasets. This combination of explicit geometry, editable components, and dynamic rendering has significant implications for real-time dynamic scene editing and content creation in 3D, with practical applicability in animated scenes and visual effects.

Abstract

One of the key advantages of 3D rendering is its ability to simulate intricate scenes accurately. One of the most widely used methods for this purpose is Gaussian Splatting, a novel approach that is known for its rapid training and inference capabilities. In essence, Gaussian Splatting involves incorporating data about the 3D objects of interest into a series of Gaussian distributions, each of which can then be depicted in 3D in a manner analogous to traditional meshes. It is regrettable that the use of Gaussians in Gaussian Splatting is currently somewhat restrictive due to their perceived linear nature. In practice, 3D objects are often composed of complex curves and highly nonlinear structures. This issue can to some extent be alleviated by employing a multitude of Gaussian components to reflect the complex, nonlinear structures accurately. However, this approach results in a considerable increase in time complexity. This paper introduces the concept of negative Gaussians, which are interpreted as items with negative colors. The rationale behind this approach is based on the density distribution created by dividing the probability density functions (PDFs) of two Gaussians, which we refer to as Diff-Gaussian. Such a distribution can be used to approximate structures such as donut and moon-shaped datasets. Experimental findings indicate that the application of these techniques enhances the modeling of high-frequency elements with rapid color transitions. Additionally, it improves the representation of shadows. To the best of our knowledge, this is the first paper to extend the simple elipsoid shapes of Gaussian Splatting to more complex nonlinear structures.

Figures (16)

• Figure 1: D-MiSo model parameterized dynamic scenes by Triangle Soup (disjoint triangles cloud), which allows modification of objects during time.
• Figure 2: Each object using the D-MiSo model is represented by Core-Gaussians and Sub-Gaussians, which form Multi-Gaussians. Each Gaussian is related to a triangle using parameterization proposed in GaMeS waczynska2024games. Triangles define the Gaussian shape (i.e., location, scale, rotation), and triangles clouds form Triangles Soups.
• Figure 3: D-MiSo allows us to modify scenes in similar ways as classical mesh-based models.
• Figure 4: Comparison of possible modifications in D-MiSo and the SC-GS. In the latter, authors use nodes while D-MiSo apply Sub-Triangle Soup (see the second column). We also must add static (pink) and dynamic (yellow) points in SC-GS to obtain modification by editing dynamic points. In practice, we have to use many static points to stop artifacts. Moreover, SC-GS is not an affine invariant and produces space when we change the size of the objects. In the case of D-MiSo, we marked points and applied modifications. Our model is superior in handling object scaling.
• Figure 5: One way to modify the object at the selected time $t_i$ is to take Core-Gaussians and apply a meshing strategy to obtain the correct mesh instead of Triangle Soup. Then, we can parametrize Sub-Gaussian in the coordinate system given bay mesh faces instead of Core-Triangle Soup. Finally, we can modify our mesh to obtain new modifications.