FastPhysGS: Accelerating Physics-based Dynamic 3DGS Simulation via Interior Completion and Adaptive Optimization

TL;DR

FastPhysGS tackles the challenge of producing physically plausible, real-time dynamic 3D Gaussian Splatting (3DGS) by introducing a two-stage framework: Instance-aware Particle Filling (IPF) to fill hollow interiors with distinct, instance-aware geometry, and Bidirectional Graph Decoupling Optimization (BGDO) to adaptively refine material parameters predicted from visual priors. IPF leverages DBSCAN for instance segmentation, convex hulls for geometry, occupancy tests, and Monte Carlo Importance Sampling to robustly fill interior regions while preserving material distinctions. BGDO decouples forward simulation from gradient-based parameter updates, using a gradient-free forward MPM over three key frames and a compressed, log-space optimization of Young’s modulus $E$ guided by stress gradients and deformation norms. The approach achieves state-of-the-art speed and memory efficiency (about 7 GB and 1 minute) with high fidelity across multiple materials, enabling practical interactive 4D physics-aware content generation and broad real-world applicability.

Abstract

Extending 3D Gaussian Splatting (3DGS) to 4D physical simulation remains challenging. Based on the Material Point Method (MPM), existing methods either rely on manual parameter tuning or distill dynamics from video diffusion models, limiting the generalization and optimization efficiency. Recent attempts using LLMs/VLMs suffer from a text/image-to-3D perceptual gap, yielding unstable physics behavior. In addition, they often ignore the surface structure of 3DGS, leading to implausible motion. We propose FastPhysGS, a fast and robust framework for physics-based dynamic 3DGS simulation:(1) Instance-aware Particle Filling (IPF) with Monte Carlo Importance Sampling (MCIS) to efficiently populate interior particles while preserving geometric fidelity; (2) Bidirectional Graph Decoupling Optimization (BGDO), an adaptive strategy that rapidly optimizes material parameters predicted from a VLM. Experiments show FastPhysGS achieves high-fidelity physical simulation in 1 minute using only 7 GB runtime memory, outperforming prior works with broad potential applications.

Figures (17)

• Figure 1: We propose FastPhysGS, an efficient and robust physics-based dynamic 3DGS simulation framework. Input a 3DGS scene, our method requires only 7 GB of memory and completes complex dynamic simulations within 1 minute, making it practical to generate real-time 4D physics-aware dynamics.
• Figure 2: FastPhysGS supports various physical behaviors including movement, collision, tearing, rotation, swaying across diverse materials, such as sand, rubber, jelly, water and elastomers.
• Figure 3: We extract the filled points as meshes for better visualization. PhysGaussian incorrectly fills the hollow region of the wicker basket, causing the mat to warp upward, while our method achieves accurate instance-aware filling.
• Figure 4: The pipeline of our method. The first stage IPF rapidly fills the interior 3DGS particles, while MCIS is designed to identify crucial points and handle complex geometries. The second stage BGDO is proposed to adaptively optimize MPM parameters. Overall, our method generates complete and realistic 4D physical dynamics in 1 minute, showcasing great potential in practical applications.
• Figure 5: We use a contiguous memory block to store labels for tracking the dynamically varying 3DGS properties.