GASP: Gaussian Splatting for Physic-Based Simulations

TL;DR

GASP integrates Gaussian Splatting with physics engines by representing objects as GaMeS-based flat Gaussians and transforming them to triangle soup for point-based physics, then back to Gaussians. It introduces a continuum-mechanics constitutive model, Material Point Method for dynamics, Gaussian control rules to suppress artifacts, and a Gaussian hierarchy to accelerate simulations. The approach operates without modifying the underlying physics engine, and demonstrates stable, high-fidelity simulations across static and dynamic scenes, including multi-object interactions, with quantitative gains in speed over baseline methods. The work demonstrates practical integration with Genesis, Blender, and Taichi Elements, broadening the applicability of GS in physics-based 3D animation and simulation.

Abstract

Physics simulation is paramount for modeling and utilizing 3D scenes in various real-world applications. However, integrating with state-of-the-art 3D scene rendering techniques such as Gaussian Splatting (GS) remains challenging. Existing models use additional meshing mechanisms, including triangle or tetrahedron meshing, marching cubes, or cage meshes. Alternatively, we can modify the physics-grounded Newtonian dynamics to align with 3D Gaussian components. Current models take the first-order approximation of a deformation map, which locally approximates the dynamics by linear transformations. In contrast, our GS for Physics-Based Simulations (GASP) pipeline uses parametrized flat Gaussian distributions. Consequently, the problem of modeling Gaussian components using the physics engine is reduced to working with 3D points. In our work, we present additional rules for manipulating Gaussians, demonstrating how to adapt the pipeline to incorporate meshes, control Gaussian sizes during simulations, and enhance simulation efficiency. This is achieved through the Gaussian grouping strategy, which implements hierarchical structuring and enables simulations to be performed exclusively on selected Gaussians. The resulting solution can be integrated into any physics engine that can be treated as a black box. As demonstrated in our studies, the proposed pipeline exhibits superior performance on a diverse range of benchmark datasets designed for 3D object rendering. The project webpage, which includes additional visualizations, can be found at https://waczjoan.github.io/GASP.

Figures (13)

• Figure 1: GASP integrates Gaussian Splatting (GS) with a physics engine to generate realistic simulations. Essentially, GASP utilizes a set of triangles to parameterize Gaussian components. This approach reduces modifying Gaussian components to 3D point clouds (consisting of vertices of triangles) processing, enabling efficient and rapid rendering. GASP can model the interaction of objects and work with dynamic scenes.
• Figure 2: The GASP pipeline is versatile and compatible with various engines, requiring no significant modifications for integration. We demonstrate its capabilities with simulations performed using the Genesis engine, Blender software, and Taichi Elements.
• Figure 3: GASP produces physical simulations using flat GS representation. It works with static GS models like GaMeS waczynska2024games or dynamic ones like D-MiSo waczynska2024d. First, each flat Gaussian component is converted to three points (a triangle face). Then, the physics engine is run on a selected point cloud to obtain trajectories for our 3D model. Finally, we convert points into Gaussian components by reverse GaMeS parametrization. Notably, we also provided some additional rules that can be applied to control Gaussians during simulations.
• Figure 4: In the GASP pipeline, the hierarchical organization of Gaussians through clustering methods allows simulations to focus only on Gaussians at the highest levels of the hierarchy. This approach speeds up computations, especially when using particle-based engines such as Taichi.
• Figure 5: Visual comparison of gravity simulations with sand-like material represented by points. First row: direct application of physical simulators to Gaussians obtained from 2DGS, which produces noticeable artifacts. Second row: application of physical simulators on triangle faces obtained using the GaMeS parametrization, the pointed artifacts and holes are not as apparent.