Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools

Xingyu Lin, Zhiao Huang, Yunzhu Li, Joshua B. Tenenbaum, David Held, Chuang Gan

TL;DR

This work tackles long-horizon deformable-object manipulation with tools by marrying differentiable physics-based gradient optimization with vision-based planning. It introduces DiffSkill, which first extracts short-horizon skills via gradient-based optimization in a differentiable simulator, then trains a neural skill abstractor to imitate these skills from RGB-D observations, and finally plans over latent intermediate goals and discrete tool choices to execute sequences of skills. Experiments on dough manipulation tasks show that DiffSkill outperforms model-free reinforcement learning baselines and a pure trajectory-optimizer, demonstrating effective planning over learned skills from visual input. The approach marks a step toward scalable, differentiable-planning-based robotic manipulation from sensory data, with potential extensions to more complex deformable objects and real-world deployment.

Abstract

We consider the problem of sequential robotic manipulation of deformable objects using tools. Previous works have shown that differentiable physics simulators provide gradients to the environment state and help trajectory optimization to converge orders of magnitude faster than model-free reinforcement learning algorithms for deformable object manipulation. However, such gradient-based trajectory optimization typically requires access to the full simulator states and can only solve short-horizon, single-skill tasks due to local optima. In this work, we propose a novel framework, named DiffSkill, that uses a differentiable physics simulator for skill abstraction to solve long-horizon deformable object manipulation tasks from sensory observations. In particular, we first obtain short-horizon skills using individual tools from a gradient-based optimizer, using the full state information in a differentiable simulator; we then learn a neural skill abstractor from the demonstration trajectories which takes RGBD images as input. Finally, we plan over the skills by finding the intermediate goals and then solve long-horizon tasks. We show the advantages of our method in a new set of sequential deformable object manipulation tasks compared to previous reinforcement learning algorithms and compared to the trajectory optimizer.

DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools

TL;DR

This work tackles long-horizon deformable-object manipulation with tools by marrying differentiable physics-based gradient optimization with vision-based planning. It introduces DiffSkill, which first extracts short-horizon skills via gradient-based optimization in a differentiable simulator, then trains a neural skill abstractor to imitate these skills from RGB-D observations, and finally plans over latent intermediate goals and discrete tool choices to execute sequences of skills. Experiments on dough manipulation tasks show that DiffSkill outperforms model-free reinforcement learning baselines and a pure trajectory-optimizer, demonstrating effective planning over learned skills from visual input. The approach marks a step toward scalable, differentiable-planning-based robotic manipulation from sensory data, with potential extensions to more complex deformable objects and real-world deployment.

Abstract

We consider the problem of sequential robotic manipulation of deformable objects using tools. Previous works have shown that differentiable physics simulators provide gradients to the environment state and help trajectory optimization to converge orders of magnitude faster than model-free reinforcement learning algorithms for deformable object manipulation. However, such gradient-based trajectory optimization typically requires access to the full simulator states and can only solve short-horizon, single-skill tasks due to local optima. In this work, we propose a novel framework, named DiffSkill, that uses a differentiable physics simulator for skill abstraction to solve long-horizon deformable object manipulation tasks from sensory observations. In particular, we first obtain short-horizon skills using individual tools from a gradient-based optimizer, using the full state information in a differentiable simulator; we then learn a neural skill abstractor from the demonstration trajectories which takes RGBD images as input. Finally, we plan over the skills by finding the intermediate goals and then solve long-horizon tasks. We show the advantages of our method in a new set of sequential deformable object manipulation tasks compared to previous reinforcement learning algorithms and compared to the trajectory optimizer.
Paper Structure (16 sections, 2 equations, 3 figures, 4 tables, 1 algorithm)

This paper contains 16 sections, 2 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Problem formulation
  4. Collecting Demonstration Trajectories with Differentiable Physics
  5. Neural Skill Abstraction
  6. Long-horizon Planning with Skill Abstraction
  7. Experiments
  8. Experimental setup
  9. Implementation Details of DiffSkill
  10. Baselines
  11. Result Analysis
  12. Ablation Analysis
  13. Related Work
  14. Conclusions
  15. Implementation Details
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Humans use various tools to manipulate deformable objects much more effectively than state-of-the-art robotic systems. This work aims to narrow the gap and develop a method named DiffSkill that learns to use tools like a rolling pin, spatula, knife, etc., to accomplish complicated dough manipulation tasks. Our method learns skill abstraction using a differentiable physics simulator, composes the skills for long-horizon manipulation of the dough, and evalulated in three challenging sequential deformable object manipulations tasks: LiftSpread, GatherTransport, and CutRearrange.
  • Figure 2: (a) Collecting demonstration trajectories by running a gradient-based trajectory optimizer in a differentiable simulator. (b) Neural abstraction by imitating the expert demonstration, which consist of a goal-conditioned policy, a feasibility predictor and a reward predictor. (c) Planning for both skill combination and the intermediate goals to solve long-horizon tasks.
  • Figure 3: Visualization of the generated plan and the corresponding execution. The plan generated by DiffSkill is shown in the left, where the first and the last image are the given initial and goal observation and in between are the generated intermediate goals. The color blocks indicate which tool is needed to reach the sub-goal. The right shows sampled frames during the execution of the generated plan using the corresponding goal conditioned policy. The numbers on the bottom right shows the achieved normalized improvement metric at that time.