SoftMAC: Differentiable Soft Body Simulation with Forecast-based Contact Model and Two-way Coupling with Articulated Rigid Bodies and Clothes

TL;DR

SoftMAC addresses the challenge of differentiable simulation for heterogeneous robotic manipulation by unifying soft bodies, articulated rigid bodies, and clothes. Our approach uses MLS-MPM for soft materials, a forecast-based contact model to reduce penetration with stable gradients, and a local penetration-tracing method to enable soft-cloth coupling, all integrated through a two-way differentiable dynamics coupling. The framework supports explicit force transfer between modalities and gradient flow across the computation graph, enabling gradient-based optimization of actions for complex tasks. Experiments on pouring, folding tacos, and bidirectional soft-rigid/soft-cloth interactions validate the accuracy and usefulness of the differentiable pipeline, highlighting practical benefits for robotic manipulation with diverse materials.

Abstract

Differentiable physics simulation provides an avenue to tackle previously intractable challenges through gradient-based optimization, thereby greatly improving the efficiency of solving robotics-related problems. To apply differentiable simulation in diverse robotic manipulation scenarios, a key challenge is to integrate various materials in a unified framework. We present SoftMAC, a differentiable simulation framework that couples soft bodies with articulated rigid bodies and clothes. SoftMAC simulates soft bodies with the continuum-mechanics-based Material Point Method (MPM). We provide a novel forecast-based contact model for MPM, which effectively reduces penetration without introducing other artifacts like unnatural rebound. To couple MPM particles with deformable and non-volumetric clothes meshes, we also propose a penetration tracing algorithm that reconstructs the signed distance field in local area. Diverging from previous works, SoftMAC simulates the complete dynamics of each modality and incorporates them into a cohesive system with an explicit and differentiable coupling mechanism. The feature empowers SoftMAC to handle a broader spectrum of interactions, such as soft bodies serving as manipulators and engaging with underactuated systems. We conducted comprehensive experiments to validate the effectiveness and accuracy of the proposed differentiable pipeline in downstream robotic manipulation applications. Supplementary materials and videos are available on our project website at https://minliu01.github.io/SoftMAC.

Figures (7)

• Figure 1: To pour water into a glass in differentiable physics simulation, we need realistic contact model for soft-rigid coupling and correct gradient calculation. Then same applies to the interactions between soft bodies and clothes.
• Figure 2: (a) Pour water into a thin glass. Grid-based model (left) leads to severe penetration. Particle-based model (middle) also causes a few particles to penetrate the glass. Forecast-based model (right) achieves the most robust performance. (b) Drag four corners of a towel to squash a plasticine. Towel goes through the plasticine without penetration tracing (left). Both the plasticine and towel deform due to the contact after adding the algorithm (right).
• Figure 3: Illustration of the contact models. Grid-based model directly computes SDF on grid nodes. Particle-based model computes SDF on particles and applies penalty on them. Forecast-based model takes a forecast step to compute SDF on particles and then adjust grid velocity accordingly.
• Figure 4: Given a target face (yellow edge), we search for the triangles in its neighboring area (red). If a particle is still in contact after system state changes, its nearest face should be within the neighboring area due to the small time interval.
• Figure 5: Computation graph for soft-rigid coupled system (the same in soft-cloth coupling). Blue nodes: state of MPM simulation. Orange nodes: state of articulated rigid body simulation. Green nodes: actions and external forces. A line is connected between two nodes if and only if we directly use one state or action to compute the resulting state. The gray box illustrates the details of forecast-based contact model.