Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map

Lei Su, Zhijie Peng, Renyuan Ren, Shengping Mao, Juan Du, Kaifeng Zhang, Xuezhou Zhu

TL;DR

This work presents Tacmap, a high-fidelity, computationally efficient tactile simulation framework anchored in volumetric penetration depth, and proves the utility of Tacmap through an in-hand rotation task, where a policy trained exclusively in simulation achieves zero-shot transfer to a physical robot.

Abstract

Vision-Based Tactile Sensors (VBTS) are essential for achieving dexterous robotic manipulation, yet the tactile sim-to-real gap remains a fundamental bottleneck. Current tactile simulations suffer from a persistent dilemma: simplified geometric projections lack physical authenticity, while high-fidelity Finite Element Methods (FEM) are too computationally prohibitive for large-scale reinforcement learning. In this work, we present Tacmap, a high-fidelity, computationally efficient tactile simulation framework anchored in volumetric penetration depth. Our key insight is to bridge the tactile sim-to-real gap by unifying both domains through a shared deform map representation. Specifically, we compute 3D intersection volumes as depth maps in simulation, while in the real world, we employ an automated data-collection rig to learn a robust mapping from raw tactile images to ground-truth depth maps. By aligning simulation and real-world in this unified geometric space, Tacmap minimizes domain shift while maintaining physical consistency. Quantitative evaluations across diverse contact scenarios demonstrate that Tacmap's deform maps closely mirror real-world measurements. Moreover, we validate the utility of Tacmap through an in-hand rotation task, where a policy trained exclusively in simulation achieves zero-shot transfer to a physical robot.

Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map

TL;DR

This work presents Tacmap, a high-fidelity, computationally efficient tactile simulation framework anchored in volumetric penetration depth, and proves the utility of Tacmap through an in-hand rotation task, where a policy trained exclusively in simulation achieves zero-shot transfer to a physical robot.

Abstract

Vision-Based Tactile Sensors (VBTS) are essential for achieving dexterous robotic manipulation, yet the tactile sim-to-real gap remains a fundamental bottleneck. Current tactile simulations suffer from a persistent dilemma: simplified geometric projections lack physical authenticity, while high-fidelity Finite Element Methods (FEM) are too computationally prohibitive for large-scale reinforcement learning. In this work, we present Tacmap, a high-fidelity, computationally efficient tactile simulation framework anchored in volumetric penetration depth. Our key insight is to bridge the tactile sim-to-real gap by unifying both domains through a shared deform map representation. Specifically, we compute 3D intersection volumes as depth maps in simulation, while in the real world, we employ an automated data-collection rig to learn a robust mapping from raw tactile images to ground-truth depth maps. By aligning simulation and real-world in this unified geometric space, Tacmap minimizes domain shift while maintaining physical consistency. Quantitative evaluations across diverse contact scenarios demonstrate that Tacmap's deform maps closely mirror real-world measurements. Moreover, we validate the utility of Tacmap through an in-hand rotation task, where a policy trained exclusively in simulation achieves zero-shot transfer to a physical robot.
Paper Structure (14 sections, 1 equation, 6 figures, 1 table)

This paper contains 14 sections, 1 equation, 6 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. METHOD
  4. Overview
  5. Deform Map Generation in Simulation
  6. Deform Map Generation in Real-World
  7. IMPLEMENTATION
  8. EXPERIMENT
  9. Experimental Setup
  10. Empirical Study
  11. Computational Efficiency
  12. Sim-to-Real Application
  13. DISCUSSION
  14. CONCLUSION

Figures (6)

  • Figure 1: (a-b) Standard setups for sim-to-real gap evaluation in (a) simulation and (b) the real world. (c) Comparison between simulation and real-world deform maps. (d-e) Deployment of an RL-based ball rotation policy from (d) simulation to (e) the real world. (f) Replaying real-world interaction data within the simulation environment for verification.
  • Figure 2: Overview of the Tacmap Framework. (a) Diagram of deform map generation in simulation and (b) diagram of deform map generation in real-world.
  • Figure 3: The implementation framework of our Tacmap in Isaac Lab and MuJoCo.
  • Figure 4: Net force comparison between simulation and real-world under the same relative contact positions between object and fingertip (left: cylinder, right: square).
  • Figure 5: Visualization of deform map across simulation and real world under the same contact position with cylinder and square.
  • ...and 1 more figures