HydroShear: Hydroelastic Shear Simulation for Tactile Sim-to-Real Reinforcement Learning

TL;DR

HydShear is a non-holonomic hydroelastic tactile simulator that advances the state-of-the-art by modeling: a) stick-slip transitions, b) path-dependent force and shear build up, and c) full SE(3) object-sensor interactions.

Abstract

In this paper, we address the problem of tactile sim-to-real policy transfer for contact-rich tasks. Existing methods primarily focus on vision-based sensors and emphasize image rendering quality while providing overly simplistic models of force and shear. Consequently, these models exhibit a large sim-to-real gap for many dexterous tasks. Here, we present HydroShear, a non-holonomic hydroelastic tactile simulator that advances the state-of-the-art by modeling: a) stick-slip transitions, b) path-dependent force and shear build up, and c) full SE(3) object-sensor interactions. HydroShear extends hydroelastic contact models using Signed Distance Functions (SDFs) to track the displacements of the on-surface points of an indenter during physical interaction with the sensor membrane. Our approach generates physics-based, computationally efficient force fields from arbitrary watertight geometries while remaining agnostic to the underlying physics engine. In experiments with GelSight Minis, HydroShear more faithfully reproduces real tactile shear compared to existing methods. This fidelity enables zero-shot sim-to-real transfer of reinforcement learning policies across four tasks: peg insertion, bin packing, book shelving for insertion, and drawer pulling for fine gripper control under slip. Our method achieves a 93% average success rate, outperforming policies trained on tactile images (34%) and alternative shear simulation methods (58%-61%).

Figures (16)

• Figure 1: Illustration of the teacher-student RL training using Asymmetric Actor-Critic Distillation (AACD)tacsl. Stage 1: a teacher actor-critic is trained with access to privileged states such as contact forces and object poses. Stage 2: the critic is initialized with the expert critic pretrained from Stage 1 and the student actor that takes high-dimensional inputs (EE pose, relative EE-goal pose, left and right tactile shear) is trained from scratch. The actor-critic is optimized with the PPO RL objective and uses encoder-LSTM-MLP networks (see Appx. \ref{['appendix:network_arch']}). Stage 3: we deploy the student actor in the real world.
• Figure 2: Illustration of HydroShear and its pipeline for tactile shear simulation. HydroShear simulates the tactile shear feedback that arises from the physical interaction between the indenter $I$ in (a) and the sensor elastomer $E$ in (b). The goal is to compute the marker displacement fields across the tactile grid query points on the elastomer $\{ \mathbf{p}_i \}_{i=1}^N$ from (b). HydroShear computes the dilation displacement field $\mathbf{M}_t^d$ in (c) and the shear displacement field $\mathbf{M}_t^s$ in (d) to get the total marker displacement field $\mathbf{M}_t$ in (e). The dilation field is computed by identifying the tactile grid points that are in contact with the indenter SDF ($\phi_I(\mathbf{p}_i) < 0$). Here, the red circle represents the outline of the contact patch. For the shear field, we take the history of indenter poses in the elastomer frame ${}^E\mathbf{X}^I_{0:t}$ to track the 3D displacement $-\hat{\mathbf{f}}_{j,t}$ of the indenter on-surface points $\{\mathbf{o}_{j,t}\}_{j=1}^M$, represented by the red arrows in (d), which connects the initial contact location of indenter on-surface points ($\{\mathbf{\hat{o}}_{j,t}\}_{j=1}^M$) to the current position of the on-surface indenter points while in-penetration to the elastomer SDF ($\phi_E(\mathbf{o}_t) < 0$).
• Figure 3: Digital twin calibration setup: In the real-world, a KUKA robot arm equipped with a sphere indenter to dilate and shear the elastomer of the GelSight Mini vision-based tactile sensor mounted on the table. We replicate the same motion in a digital twin of the real-world setup to run baseline and our algorithms to calibrate the tactile simulation model.
• Figure 4: Different randomization modes per task. For Peg Insertion, we randomize the socket position and the in-hand pose of the peg. For Bin Packing, we randomize the goal location out of 16 cube locations and randomize the amount of squish, its direction (horizontal or vertical), and the number of cubes (single or double) used to squish the goal space. For Book Shelving, we randomize the books that are neighboring the goal book pose. We either squish the goal by translating neighboring books in parallel or by tilting to occlude the goal region. For Drawer Pulling, we randomize the force and timing at which perturbation is applied to induce slippage as the robot policy pulls the drawer out.
• Figure 5: Teacher RL training curves. We train the teacher actor-critic over two stages: one without contact penalty and one with contact penalty. The discontinuity is from this finetuning.