Fine Manipulation Using a Tactile Skin: Learning in Simulation and Sim-to-Real Transfer

TL;DR

A novel model of a tactile skin that can be used together with rigid-body physics simulators and shows in simulation experiments that tactile feedback is crucial for precise manipulation and reaching sub-taxel resolution of <1 mm (despite a taxel spacing of 4 mm).

Abstract

We want to enable fine manipulation with a multi-fingered robotic hand by using modern deep reinforcement learning methods. Key for fine manipulation is a spatially resolved tactile sensor. Here, we present a novel model of a tactile skin that can be used together with rigid-body (hence fast) physics simulators. The model considers the softness of the real fingertips such that a contact can spread across multiple taxels of the sensor depending on the contact geometry. We calibrate the model parameters to allow for an accurate simulation of the real-world sensor. For this, we present a self-contained calibration method without external tools or sensors. To demonstrate the validity of our approach, we learn two challenging fine manipulation tasks: Rolling a marble and a bolt between two fingers. We show in simulation experiments that tactile feedback is crucial for precise manipulation and reaching sub-taxel resolution of < 1 mm (despite a taxel spacing of 4 mm). Moreover, we demonstrate that all policies successfully transfer from the simulation to the real robotic hand.

Figures (10)

• Figure 1: Time sequence of a fine manipulation task ($\Delta t = 1s$ between images). The marble has to be rolled between two fingers along a specified trajectory (here a circle) using the feedback from the tactile skin (blue fingertip, see also \ref{['fig:tactile_sensor']}, Right). The lower row shows the actual tactile image with $4\times 4$ taxels. The task is executed on the torque-controlled DLR-Hand II Butterfass2001 which is mounted on the humanoid robot DLR Agile Justin Bauml.2014 (on the right). Only the hand's position and torque sensors and the tactile skin are used, but no, e.g., visual input.
• Figure 2: Left: Tactile response due to soft contact. The top row shows two contact situations of a bolt with a tactile sensor on a cylindrical fingertip. Only if the contact is soft, the bolt will penetrate the fingertip resulting in a contact area causing multi-taxel response instead of a single contact point. The second row depicts the pressure distribution on the fingertip and the bottom row the resulting response for the individual taxel. Right: Tactile sensor. The top row shows the sensor without and with the rubber glove covering (for better grip). The sensor has $4\times 4$ taxels, which each measure $2.5mm \times 2.5mm$ and are arranged on a $4mm$ grid, leading to a 1.5mm in between. At the bottom, details of the sensor and contact modeling are depicted as needed in \ref{['sec:skin_model']}. The discretization of the sensor is shown for an exemplary resolution of $0.5mm$ per tactile point. Red tactile points correspond to the taxel regions $\mathcal{T}_j$, whereas blue points lie within the gaps and purple points form a margin around the sensor. These additional points are needed to correctly model the contact at each point of the sensor.
• Figure 3: Visualization of the tactile skin model as detailed in \ref{['sec:skin_model']}.
• Figure 4: Left: Self-Contained data generation. The top image shows the cap with an spherical indenter of radius $6mm$. The bottom image shows the set of $60$ desired contact points, randomly selected from a uniform grid with a resolution of $1mm$. While the desired contact points are shown in black, the measured contact points are shown in red. Moreover, the slack introduced for each measured contact point is visualized. Right: Results of the micro variable optimization for an exemplary set of macro variables. The green box marks a positive result where the images match well between simulation and reality. The red box marks a negative example that can be caused if the slack of $\pm2mm$ is not sufficient.
• Figure 5: Calibration scheme.