TacSL: A Library for Visuotactile Sensor Simulation and Learning

TL;DR

TacSL introduces a GPU-accelerated visuotactile sensor simulator integrated with Isaac, dramatically speeding up tactile image and force-field generation to enable end-to-end tactile-policy learning. It provides a learning toolkit with offline/online policy distillation and reinforcement learning via PPO, plus the novel AACD algorithm that leverages a pretrained critic for efficient high-dimensional policy training. A sim-to-real transfer recipe combines soft-contact parameter randomization and tactile image augmentation, enabling zero-shot transfer to real robots for tasks like peg placement and insertion. Empirical results show substantial speedups over CPU baselines, effective online distillation and AACD in simulation, and successful real-world transfer, demonstrating TacSL as a practical testbed for visuotactile learning in contact-rich manipulation.

Abstract

For both humans and robots, the sense of touch, known as tactile sensing, is critical for performing contact-rich manipulation tasks. Three key challenges in robotic tactile sensing are 1) interpreting sensor signals, 2) generating sensor signals in novel scenarios, and 3) learning sensor-based policies. For visuotactile sensors, interpretation has been facilitated by their close relationship with vision sensors (e.g., RGB cameras). However, generation is still difficult, as visuotactile sensors typically involve contact, deformation, illumination, and imaging, all of which are expensive to simulate; in turn, policy learning has been challenging, as simulation cannot be leveraged for large-scale data collection. We present TacSL (taxel), a library for GPU-based visuotactile sensor simulation and learning. TacSL can be used to simulate visuotactile images and extract contact-force distributions over $200\times$ faster than the prior state-of-the-art, all within the widely-used Isaac Simulator. Furthermore, TacSL provides a learning toolkit containing multiple sensor models, contact-intensive training environments, and online/offline algorithms that can facilitate policy learning for sim-to-real applications. On the algorithmic side, we introduce a novel online reinforcement-learning algorithm called asymmetric actor-critic distillation (AACD), designed to effectively and efficiently learn tactile-based policies in simulation that can transfer to the real world. Finally, we demonstrate the utility of our library and algorithms by evaluating the benefits of distillation and multimodal sensing for contact-rich manipulation tasks, and most critically, performing sim-to-real transfer. Supplementary videos and results are at https://iakinola23.github.io/tacsl/.

Figures (17)

• Figure 1: Using state-of-the-art tactile simulation methods, TacSL equips a simulated robot (left) with tactile-sensing capabilities that mirror those available on a real-world robot (right). By employing algorithms provided within the TacSL learning toolkit, tactile-based policies for contact-rich tasks (e.g., peg insertion) are trained within simulation, thus enabling scalable data collection and preserving the lifespan of the real-world tactile sensor. Subsequently, learned policies can be transferred successfully to the real-world system.
• Figure 2: TacSL has 3 main components: (Left) A fast visuotactile simulation module that produces tactile images and force fields. (Top-right) A set of sensors and manipulation environments for tactile-based policy learning. (Bottom-right) Offline and online distillation as well as reinforcement learning algorithms to facilitate sim-to-real transfer.
• Figure 3: Illustration of the soft contact model. Objects are modeled as rigid bodies, interpenetration constraints are relaxed for the soft object, and a level of interpenetration is allowed according to a spring-damper system. Right: the level of interpenetration scales with magnitude of the applied force.
• Figure 4: Tactile Policy Distillation is a method to efficiently train a policy that takes as input high-dimensional visuotactile observations. An expert policy $\pi_{e}$ is first trained to solve the task; this policy takes as input privileged state information available only in the simulation (e.g., contact forces) and predicts action $a_e$. During distillation, a student policy is trained to imitate the expert policy; this policy takes as input observations that are available in both simulation and the real world (e.g., visuotactile images) and predicts action $a_s$. The expert action $a_e$ is always used as training data for the student action $a_s$. Nevertheless, to advance the simulator to the next step, an action is sampled from either the expert policy or the student policy.
• Figure 5: Asymmetric Actor Critic Distillation (AACD): Illustration of the two stages of AACD. In the first stage, an expert agent (actor and critic) is trained using RL to learn the task using privileged information available in simulation. In the second stage, the critic is initialized with the pretrained "expert" critic. The high-dimensional student policy is similarly trained using RL, as the critic is fine-tuned. This approach retains the performance benefits of RL to acquire high dimensional policies with reward-maximizing behaviors.