A Low-Cost Vision-Based Tactile Gripper with Pretraining Learning for Contact-Rich Manipulation

TL;DR

This work introduces LVTG, a low-cost vision-based tactile gripper with a large sensing area and modular, wear-resistant skin designed for stable contact-rich manipulation. It leverages a CLIP-inspired contrastive pretraining objective to align tactile and visual embeddings, enabling cross-modal perception that improves the ACT policy in tactile-rich tasks. Experiments demonstrate superior grasp stability and significantly improved policy performance when tactile feedback and pretraining are used, along with excellent durability and rapid replacement advantages. The approach offers a practical, scalable path to widespread deployment of visuo-tactile manipulation in real-world settings, including smart agriculture and industrial automation.

Abstract

Robotic manipulation in contact-rich environments remains challenging, particularly when relying on conventional tactile sensors that suffer from limited sensing range, reliability, and cost-effectiveness. In this work, we present LVTG, a low-cost visuo-tactile gripper designed for stable, robust, and efficient physical interaction. Unlike existing visuo-tactile sensors, LVTG enables more effective and stable grasping of larger and heavier everyday objects, thanks to its enhanced tactile sensing area and greater opening angle. Its surface skin is made of highly wear-resistant material, significantly improving durability and extending operational lifespan. The integration of vision and tactile feedback allows LVTG to provide rich, high-fidelity sensory data, facilitating reliable perception during complex manipulation tasks. Furthermore, LVTG features a modular design that supports rapid maintenance and replacement. To effectively fuse vision and touch, We adopt a CLIP-inspired contrastive learning objective to align tactile embeddings with their corresponding visual observations, enabling a shared cross-modal representation space for visuo-tactile perception. This alignment improves the performance of an Action Chunking Transformer (ACT) policy in contact-rich manipulation, leading to more efficient data collection and more effective policy learning. Compared to the original ACT method, the proposed LVTG with pretraining achieves significantly higher success rates in manipulation tasks.

Figures (7)

• Figure 1: The figure of LVTG's form factor.
• Figure 2: (a) The exploded mechanical view of LVTG. (b) The illustration of the LVTG Optical Path.
• Figure 3: The pipeline of tactile image enhancement.
• Figure 4: The pipeline of manipulation learning.
• Figure 5: Tactile perception results during grasping different objects.