Large-scale Deployment of Vision-based Tactile Sensors on Multi-fingered Grippers

TL;DR

This paper addresses large-scale, multi-surface tactile sensing on multi-finger grippers by introducing a synchronized image acquisition system, a compact modular VBTS design, and a zero-shot calibration method. The approach enables seven VBTS to operate simultaneously on a three-finger GelGripper across finger phalanges and the palm, delivering high-resolution tactile feedback while reducing calibration data requirements. The zero-shot calibration uses an MLP to map RGB intensities and local gradients to surface depth, and the synchronized acquisition minimizes latency and ensures reliability across sensors, as validated in grasp and manipulation tasks. The work has practical implications for robust, dexterous manipulation in robotics, enabling scalable tactile sensing across complex gripper morphologies.

Abstract

Vision-based Tactile Sensors (VBTSs) show significant promise in that they can leverage image measurements to provide high-spatial-resolution human-like performance. However, current VBTS designs, typically confined to the fingertips of robotic grippers, prove somewhat inadequate, as many grasping and manipulation tasks require multiple contact points with the object. With an end goal of enabling large-scale, multi-surface tactile sensing via VBTSs, our research (i) develops a synchronized image acquisition system with minimal latency,(ii) proposes a modularized VBTS design for easy integration into finger phalanges, and (iii) devises a zero-shot calibration approach to improve data efficiency in the simultaneous calibration of multiple VBTSs. In validating the system within a miniature 3-fingered robotic gripper equipped with 7 VBTSs we demonstrate improved tactile perception performance by covering the contact surfaces of both gripper fingers and palm. Additionally, we show that our VBTS design can be seamlessly integrated into various end-effector morphologies significantly reducing the data requirements for calibration.

Figures (9)

• Figure 1: Large-scale vbts deployment on a robotic gripper. (a) Acquiring rich tactile information utilizing the three-fingered robotic gripper. Seven modular vbts are deployed into the phalanges of each finger as well as the palm of the gripper. (b)(c) Alternative configurations using modular vbts for other tasks.
• Figure 2: Synchronized image acquisition system. Multiple sensor boards are cascade-connected to the hub board through FFC cables. The hub board provides the power supply for both the sensor boards and the motors, and handles the communication between the acquisition system and the host PC.
• Figure 3: Sequence diagram of the acquisition process. The system is designed to minimize synchronization error, and thus provide a high frame rate with limited latency.
• Figure 4: Design and fabrication of the modular vision-based tactile sensor. (a) The structure of the vbts consists of a modular image acquisition board, a wide-lens camera, an illumination system, a supporting structure, a contact module, and a shell. (b) The view of the wide-lens camera covers the whole surface of the contact module. (c) The internal area lighting system utilizes side illumination towards the supporting structure for shade accentuation and compactness. (d) The elastomer serves as the contact module to provide super-high spatial resolution. (e) The overall dimensions of each sensor are minimized to $28 \times 20 \times 19~\mathrm{mm}$.
• Figure 5: The zero-short calibration process for large-scale deployment of vbts. To jointly calibrate the vbts that share the similiar configuration, a multiple-layer perception (MLP) neural network is utilized to learn the mapping between pixel intensity within RGB color channels, position information, and gradients in the x and y directions.