Gaussian-Augmented Physics Simulation and System Identification with Complex Colliders

TL;DR

AS-DiffMPM introduces a differentiable Material Point Method capable of collisions with arbitrarily shaped rigid bodies and integrates with Gaussian-based rendering for end-to-end system identification from multi-view observations. It advances collision handling with a CPIC-based Collision Grid that supports meshes or 2D Gaussian colliders, and it couples with NeRF and Gaussian splatting renderers to enable gradient-based parameter optimization from visuals. The approach demonstrates accurate recovery of Newtonian, non-Newtonian, and granular material parameters in complex collider settings and validates a real-world dough experiment, showcasing practical applicability beyond planar boundaries. Overall, the work broadens differentiable physics-based system identification to more realistic interactions, enabling richer scene synthesis and objective parameter estimation in robotics and graphics.

Abstract

System identification involving the geometry, appearance, and physical properties from video observations is a challenging task with applications in robotics and graphics. Recent approaches have relied on fully differentiable Material Point Method (MPM) and rendering for simultaneous optimization of these properties. However, they are limited to simplified object-environment interactions with planar colliders and fail in more challenging scenarios where objects collide with non-planar surfaces. We propose AS-DiffMPM, a differentiable MPM framework that enables physical property estimation with arbitrarily shaped colliders. Our approach extends existing methods by incorporating a differentiable collision handling mechanism, allowing the target object to interact with complex rigid bodies while maintaining end-to-end optimization. We show AS-DiffMPM can be easily interfaced with various novel view synthesis methods as a framework for system identification from visual observations.

Figures (9)

• Figure 1: We present the Any-Shape Differentiable Material Point Method (AS-DiffMPM), a particle-based framework for simulating collisions with arbitrarily shaped rigid bodies. When integrated with Gaussian-based rendering, it enables physically plausible animation and system identification (estimation of object physical parameters from visual observations).
• Figure 2: Overview. Given multi-view input images, we separately reconstruct the continuum object using a rendering model (e.g., sun2022directhuang20242d) and the rigid body collider with 2DGS huang20242d. Particle trajectories are subsequently advected using AS-DiffMPM. Finally, the updated particle positions are mapped back to rendering primitives for photorealistic and physically plausible animation.
• Figure 3: Qualitative examples of reference frames used in experiments. Note that previous works li2023packaneko2024improvingcaigiczhong2024reconstruction perform system identification only with a planar surface and do not support such colliders.
• Figure 4: Example visualization of the dough experiment.
• Figure 5: Particle trajectories of a Newtonian material colliding with the Armadillo rigid body.