PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics

TL;DR

PlasticineLab introduces the first differentiable-physics benchmark for elastoplastic soft-body manipulation, enabling gradient-based planning via a Taichi-based MLS-MPM simulator with DiffTaichi. The 10 tasks (50 configurations) test a range of manipulation primitives and reveal that gradient-based trajectory optimization can quickly find solutions, while traditional RL methods struggle on long-horizon, high-DoF tasks. The work highlights the potential of integrating differentiable physics with RL and planning, and proposes directions for improved controllers, sim-to-real transfer, and generalization. Overall, PlasticineLab provides a flexible platform to study how differentiable physics can advance complex soft-body skill learning and planning algorithms.

Abstract

Simulated virtual environments serve as one of the main driving forces behind developing and evaluating skill learning algorithms. However, existing environments typically only simulate rigid body physics. Additionally, the simulation process usually does not provide gradients that might be useful for planning and control optimizations. We introduce a new differentiable physics benchmark called PasticineLab, which includes a diverse collection of soft body manipulation tasks. In each task, the agent uses manipulators to deform the plasticine into the desired configuration. The underlying physics engine supports differentiable elastic and plastic deformation using the DiffTaichi system, posing many under-explored challenges to robotic agents. We evaluate several existing reinforcement learning (RL) methods and gradient-based methods on this benchmark. Experimental results suggest that 1) RL-based approaches struggle to solve most of the tasks efficiently; 2) gradient-based approaches, by optimizing open-loop control sequences with the built-in differentiable physics engine, can rapidly find a solution within tens of iterations, but still fall short on multi-stage tasks that require long-term planning. We expect that PlasticineLab will encourage the development of novel algorithms that combine differentiable physics and RL for more complex physics-based skill learning tasks.

Figures (6)

• Figure 1: Left: A child deforming a piece of plasticine into a thin pie using a rolling pin. Right: The challenging RollingPin scene in PlasticineLab. The agent needs to flatten the material by rolling the pin back and forth, so that the plasticine deforms into the target shape.
• Figure 2: Tasks and reference solutions of PlasticineLab. Certain tasks require multi-stage planning.
• Figure 3: The final normalized incremental IoU score achieved by RL methods within $10^4$ epochs. Scores lower than 0 are clamped. The dashed orange line indicates the theoretical upper limit.
• Figure 4: Rewards and their variances in each task w.r.t. the number of episodes spent on training. We clamp the reward to be greater than $0$ for a better illustration.
• Figure 5: Rewards w.r.t. the number of training episode on $6$ environments. Their yield stresses are 10, 20, 50, 100, 200, and 1000.