Sim-to-Real Gentle Manipulation of Deformable and Fragile Objects with Stress-Guided Reinforcement Learning

TL;DR

The paper tackles deformable and fragile object manipulation by minimizing internal stress while achieving task goals, using a vision-based reinforcement learning framework trained in simulation. It introduces a stress-penalized reward, offline demonstrations, and curriculum learning to bootstrap policy learning and stabilize sim-to-real transfer. Key contributions include a first visuomotor DFOM framework that explicitly accounts for fragility, a quadratic stress penalty combining mean and top-stress statistics, and demonstrated zero-shot transfer to real tofu manipulation with substantially reduced damage. The results suggest this stress-aware, model-free approach can generalize to soft objects without specialized tactile sensing, offering practical benefits for delicate manipulation tasks.

Abstract

Robotic manipulation of deformable and fragile objects presents significant challenges, as excessive stress can lead to irreversible damage to the object. While existing solutions rely on accurate object models or specialized sensors and grippers, this adds complexity and often lacks generalization. To address this problem, we present a vision-based reinforcement learning approach that incorporates a stress-penalized reward to discourage damage to the object explicitly. In addition, to bootstrap learning, we incorporate offline demonstrations as well as a designed curriculum progressing from rigid proxies to deformables. We evaluate the proposed method in both simulated and real-world scenarios, showing that the policy learned in simulation can be transferred to the real world in a zero-shot manner, performing tasks such as picking up and pushing tofu. Our results show that the learned policies exhibit a damage-aware, gentle manipulation behavior, demonstrating their effectiveness by decreasing the stress applied to fragile objects by 36.5% while achieving the task goals, compared to vanilla RL policies.

Figures (7)

• Figure 1: Overview of our approach to deformable and fragile object manipulation using only visual input. Our stress-guided RL policy, trained in simulation, transfers to the real world zero-shot. It enables tasks such as picking up and pushing tofu without causing damage, in contrast to a baseline. For visualization, the tofu is dyed blue in real-world experiments.
• Figure 2: Overview of our stress-guided RL framework for deformable and fragile object manipulation. Training begins with (1) curriculum learning, where the agent first learns the task in rigid simulation before switching to soft-body simulation. (2) In the soft setting, stress-based guidance encourages safe manipulation, and expert demonstrations bootstrap learning. (3) Once converged, the policy is deployed zero-shot in the real world, with the setup closely matched to the simulation. The policy takes as input the segmented object point cloud and an 11-dimensional state vector: the 7-DoF end-effector pose pose ${\mathbf{p} \in SE(3)}$, the gripper width $g \in \mathbb{R}$ (in cm), and the centroid of the point cloud $\mathbf{c}$.
• Figure 3: Global statistics (mean stress and median stress) may not reflect the local large deformation (as shown on the right), whereas max stress more reliably captures the information.
• Figure 4: Experimental setup showing the robot manipulator, the target object, and the RGB-D camera used to provide observations to the control policy.
• Figure 5: Qualitative results for the pick-up task of a cylindrical tofu. Each column illustrates a rollout with a different method. A green tick indicates task success without visible damage, while a red cross denotes either task failure or damage to the tofu.