Thin-Shell Object Manipulations With Differentiable Physics Simulations

TL;DR

The paper addresses the challenge of manipulating diverse thin-shell objects by introducing ThinShellLab, a fully differentiable simulation platform that models volumetric and thin-shell deformables with bending plasticity and frictional contact. It demonstrates that neither pure gradient-based nor pure sampling-based methods suffice for these tasks, and shows that a hybrid CMA-ES + gradient-based trajectory optimization yields robust performance across a suite of manipulation and inverse-design tasks. The framework supports real-to-sim system identification and real-world transfer, reducing the sim-to-real gap and enabling practical deployment. Overall, ThinShellLab offers a comprehensive differentiable benchmark and optimization toolkit for advancing flexible thin-shell manipulation in robotics.

Abstract

In this work, we aim to teach robots to manipulate various thin-shell materials. Prior works studying thin-shell object manipulation mostly rely on heuristic policies or learn policies from real-world video demonstrations, and only focus on limited material types and tasks (e.g., cloth unfolding). However, these approaches face significant challenges when extended to a wider variety of thin-shell materials and a diverse range of tasks. While virtual simulations are shown to be effective in diverse robot skill learning and evaluation, prior thin-shell simulation environments only support a subset of thin-shell materials, which also limits their supported range of tasks. We introduce ThinShellLab - a fully differentiable simulation platform tailored for robotic interactions with diverse thin-shell materials possessing varying material properties, enabling flexible thin-shell manipulation skill learning and evaluation. Our experiments suggest that manipulating thin-shell objects presents several unique challenges: 1) thin-shell manipulation relies heavily on frictional forces due to the objects' co-dimensional nature, 2) the materials being manipulated are highly sensitive to minimal variations in interaction actions, and 3) the constant and frequent alteration in contact pairs makes trajectory optimization methods susceptible to local optima, and neither standard reinforcement learning algorithms nor trajectory optimization methods (either gradient-based or gradient-free) are able to solve the tasks alone. To overcome these challenges, we present an optimization scheme that couples sampling-based trajectory optimization and gradient-based optimization, boosting both learning efficiency and converged performance across various proposed tasks. In addition, the differentiable nature of our platform facilitates a smooth sim-to-real transition.

Figures (7)

• Figure 1: It's common for us to interact with thin-shell materials, such as bending a paper to lift a block (a) or picking up a piece of paper (b). Aiming to boost versatile robotic skill acquiring for diverse thin-shell materials, we propose ThinShellLab, a fully differentiable simulation platform together with a set of benchmark tasks ((a) and (b) bottom). Moreover, to bring simulation in line with the real-world, we adjust physical properties by utilizing the gradient and real-world observations (c). After that, we successfully deploy our policy learned from simulation to the real world (d).
• Figure 2: This figure shows 7 manipulation tasks on the left side and 3 inverse design tasks on the right side. We display the initial and the final position for our manipulation tasks and we show the target transparently in Lifting and Forming. For Sliding, we show the final goal with transparency and show the results before and after optimization on the right column. For Bouncing, we show the final state before and after optimization in transparency. We display the behavior of Card in transparent and draw an array representing the target moving direction of the card.
• Figure 3: Reward curves for all methods. We plot the prefix maximum for the CMA-ES method. We observe that the hybrid method of CMA-ES + GD achieves the best performance in most tasks.
• Figure 4: Optimization curve of system identification tasks.
• Figure 5: Tactile dot matrix after optimization.