Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation

Zilin Si, Gu Zhang, Qingwei Ben, Branden Romero, Zhou Xian, Chao Liu, Chuang Gan

TL;DR

DIFFTACTILE is introduced, a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback, and a method to infer the optical response of the tactile sensor to contact using an efficient pixel-based neural module.

Abstract

We introduce DIFFTACTILE, a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback. In contrast to prior tactile simulators which primarily focus on manipulating rigid bodies and often rely on simplified approximations to model stress and deformations of materials in contact, DIFFTACTILE emphasizes physics-based contact modeling with high fidelity, supporting simulations of diverse contact modes and interactions with objects possessing a wide range of material properties. Our system incorporates several key components, including a Finite Element Method (FEM)-based soft body model for simulating the sensing elastomer, a multi-material simulator for modeling diverse object types (such as elastic, elastoplastic, cables) under manipulation, a penalty-based contact model for handling contact dynamics. The differentiable nature of our system facilitates gradient-based optimization for both 1) refining physical properties in simulation using real-world data, hence narrowing the sim-to-real gap and 2) efficient learning of tactile-assisted grasping and contact-rich manipulation skills. Additionally, we introduce a method to infer the optical response of our tactile sensor to contact using an efficient pixel-based neural module. We anticipate that DIFFTACTILE will serve as a useful platform for studying contact-rich manipulations, leveraging the benefits of dense tactile feedback and differentiable physics. Code and supplementary materials are available at the project website https://difftactile.github.io/.

DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation

TL;DR

DIFFTACTILE is introduced, a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback, and a method to infer the optical response of the tactile sensor to contact using an efficient pixel-based neural module.

Abstract

We introduce DIFFTACTILE, a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback. In contrast to prior tactile simulators which primarily focus on manipulating rigid bodies and often rely on simplified approximations to model stress and deformations of materials in contact, DIFFTACTILE emphasizes physics-based contact modeling with high fidelity, supporting simulations of diverse contact modes and interactions with objects possessing a wide range of material properties. Our system incorporates several key components, including a Finite Element Method (FEM)-based soft body model for simulating the sensing elastomer, a multi-material simulator for modeling diverse object types (such as elastic, elastoplastic, cables) under manipulation, a penalty-based contact model for handling contact dynamics. The differentiable nature of our system facilitates gradient-based optimization for both 1) refining physical properties in simulation using real-world data, hence narrowing the sim-to-real gap and 2) efficient learning of tactile-assisted grasping and contact-rich manipulation skills. Additionally, we introduce a method to infer the optical response of our tactile sensor to contact using an efficient pixel-based neural module. We anticipate that DIFFTACTILE will serve as a useful platform for studying contact-rich manipulations, leveraging the benefits of dense tactile feedback and differentiable physics. Code and supplementary materials are available at the project website https://difftactile.github.io/.
Paper Structure (34 sections, 3 equations, 9 figures, 15 tables)

This paper contains 34 sections, 3 equations, 9 figures, 15 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Tactile Simulation
  4. System overview
  5. Tactile sensor simulation
  6. Optical Simulation
  7. Object Simulation
  8. Penalty-based contact model
  9. Experiments
  10. Overview
  11. Simulation Setup
  12. System identification
  13. Optical Simulation
  14. Grasping
  15. Contact-rich Manipulation
  16. ...and 19 more sections

Figures (9)

  • Figure 1: Grasping a deformable object in the real world and in DiffTactile.
  • Figure 2: DiffTactile tasks. Grasping: We grasp a set of four objects with different geometries and materials. Surface following: A sensor travels on the surface while maintaining the contact. Cable straightening: A pair of sensors straighten a cable by gripping and sliding from a fixed end. Object reposing: A sensor pushes an object to let it stand against a wall. Case opening: A sensor opens the cap of a case.
  • Figure 3: System identification to optimize the FEM sensor model and contact model's physical parameters with tactile readings and force readings from the real world.
  • Figure 4: Tactile optical simulation compared with real data capturing various contact geometries.
  • Figure 5: Simulation pipeline for each simulation step. Both the FEM sensor and MPM object have their pre-contact updates, and then we use a two-way coupling to handle collision and calculate contact forces. The contact forces are used in post-contact for both the FEM sensor and MPM object.
  • ...and 4 more figures