Tacchi 2.0: A Low Computational Cost and Comprehensive Dynamic Contact Simulator for Vision-based Tactile Sensors

TL;DR

This work tackles the high cost and robustness limitations of generating tactile data for vision-based sensors by advancing Tacchi into Tacchi 2.0, a physics-based simulator that combines Material Point Method elastomer modeling with a pinhole camera projection to produce tactile images, marker motion images, and joint images. The method provides an efficient, camera-aware pathway to synthesize marker motion under pressing, slipping, and rotating, demonstrated across multiple sensors with strong cross-sensor robustness. Key contributions include a dedicated marker-motion image simulation method, full Tacchi 2.0 capability for tactile and marker data fusion, and extensive experimental validation showing low computational cost and high fidelity relative to real data. The results imply significant potential for scalable, multimodal tactile data generation to support large-scale robotic manipulation models.

Abstract

With the development of robotics technology, some tactile sensors, such as vision-based sensors, have been applied to contact-rich robotics tasks. However, the durability of vision-based tactile sensors significantly increases the cost of tactile information acquisition. Utilizing simulation to generate tactile data has emerged as a reliable approach to address this issue. While data-driven methods for tactile data generation lack robustness, finite element methods (FEM) based approaches require significant computational costs. To address these issues, we integrated a pinhole camera model into the low computational cost vision-based tactile simulator Tacchi that used the Material Point Method (MPM) as the simulated method, completing the simulation of marker motion images. We upgraded Tacchi and introduced Tacchi 2.0. This simulator can simulate tactile images, marked motion images, and joint images under different motion states like pressing, slipping, and rotating. Experimental results demonstrate the reliability of our method and its robustness across various vision-based tactile sensors.

Figures (8)

• Figure 1: The marker points move with the elastomer when pressed, and these displacements are captured by a camera and presented as images. This paper uses the MPM and a camera model to simulate the elastomer deformation and the camera's capture process, respectively. (A): Pinhole camera model schematic diagram. (B): Schematic diagram of vision-based tactile sensor. (C): Marker motion simulation effects of pressing, sliding, and rotating.
• Figure 2: Tacchi 2.0 overall flow chart. The overall flow chart of Tacchi 2.0, including the camera calibration module, Tacchi 2.0 initialization module, and Tacchi 2.0 calculate module.
• Figure 3: Schematic diagram of particle formation of elastomer and representative marker point particles for different sensors.
• Figure 4: Schematic diagram of the pinhole camera model.
• Figure 5: The mask image of marker motion images under pressing, sliding, and rotating. The sensor that we used to simulate and collect real data is designed based on ZSX24. The Size of the marker array is $9\times 9$.