Evetac: An Event-based Optical Tactile Sensor for Robotic Manipulation

TL;DR

This work introduces Evetac, an open-source event-based optical tactile sensor that replaces RGB cameras with a high-temporal-resolution event camera to enable 1000 Hz touch processing. It combines a dotted GelSight-like elastomer, a gradient-based dot-tracking algorithm, and data-driven slip detection/prediction models to support robust closed-loop grasping across diverse objects, while achieving substantial data-rate reductions compared to RGB sensors. The findings demonstrate vibration sensing up to 498 Hz, effective shear-force reconstruction from dot displacements, and high success rates in reactive grasp control, including transfer to another Evetac sensor and the closed Gel. Together, these results suggest that fast, efficient event-based tactile sensing can significantly advance human-like manipulation in robotics and enable real-time feedback control in unstructured environments.

Abstract

Optical tactile sensors have recently become popular. They provide high spatial resolution, but struggle to offer fine temporal resolutions. To overcome this shortcoming, we study the idea of replacing the RGB camera with an event-based camera and introduce a new event-based optical tactile sensor called Evetac. Along with hardware design, we develop touch processing algorithms to process its measurements online at 1000 Hz. We devise an efficient algorithm to track the elastomer's deformation through the imprinted markers despite the sensor's sparse output. Benchmarking experiments demonstrate Evetac's capabilities of sensing vibrations up to 498 Hz, reconstructing shear forces, and significantly reducing data rates compared to RGB optical tactile sensors. Moreover, Evetac's output and the marker tracking provide meaningful features for learning data-driven slip detection and prediction models. The learned models form the basis for a robust and adaptive closed-loop grasp controller capable of handling a wide range of objects. We believe that fast and efficient event-based tactile sensors like Evetac will be essential for bringing human-like manipulation capabilities to robotics. The sensor design is open-sourced at https://sites.google.com/view/evetac .

Figures (18)

• Figure 1: Two Evetac sensors installed in the ROBOTIS RH-P12-RN(A) gripper holding a pen. In the bottom left and right, we depict a magnified version of the sensors' measurements. Evetac is an open-source event-based optical tactile sensor for robotic manipulation. Its main components are an illuminated, dotted, soft silicone gel that interacts with the environment. Changes in gel configuration are captured by an event-based camera inside the sensor as shown in the bottom left & right.
• Figure 2: Exploded view of the proposed Evetac sensor. From left to right: A) DVXplorer Mini, event-based camera, B) 3D printed camera housing, C) LED stripe for illumination from the inside, and D) GelSight Mini dotted gel. The 3D printed housing allows adjusting the camera's distance from the gel to ensure that it is in focus. It also allows mounting Evetac to an external gripper (cf . Figure \ref{['fig:evetac_parallel_grip']}). For the components see Table \ref{['table:hw_componets']}. The total dimensions of the assembled sensor are $32 \mathrm{x}33 \mathrm{x} 65 \mm$ (width x height x length).
• Figure 3: Both pictures show the current contact configuration (left) & Evetac's output in image form (right). As mentioned in Sec. \ref{['sec:raw_sensor_output']}, Evetac returns the raw events accumulated for 1. Since Evetac's raw output is extremely sparse, for visualization purposes, we actually illustrate the combination of the last 5 measurements, i.e., the events triggered within the last 5. In the pictures that show Evetac's raw output, all gray pixels correspond to locations where no events have been triggered, while the white & black pixels illustrate locations of on & off-events, respectively.
• Figure 4: Illustrating how the movement of a black dot in front of a white, bright background triggers events. Left: At time $t_i$, the dot is moving to the right. Middle: This results in a slightly shifted position at time $t_{i+1}$. Right: The dot movement between $t_i$ and $t_{i+1}$ causes events $\mathcal{S}_\mathrm {E}(t_{i+1})$ that are visualized in pictorial form. The pixels colored in white correspond to the locations where on-events were triggered. The brightness of all these pixels changed from being occupied by the black dot at $t_i$ to being occupied by the white background at $t_{i+1}$. Due to this intensity change, events have been triggered at these locations. For the pixels colored in black, the opposite happened, i.e., the brightness changed from the white background to the black dot. At all the remaining pixel locations, no events have been triggered. They are thus colored in neutral grey.
• Figure 5: Experimental setups for the Evetac benchmarking experiments of sensing vibrations (cf. Sec. \ref{['sec:sensing_vibrations']}, (a)) and shear force reconstruction (cf. Sec. \ref{['sec:shear_force_reconstruction']}, (b)). For both experiments, Evetac is mounted in the end-effector of a Franka Panda robot. In Fig. \ref{['fig:vibration_sensing']}, Evetac presses against a speaker which is set to generate a tone with a desired frequency. Through making contact with the speaker, Evetac can perceive the vibrations of the speaker and reconstruct the vibration frequency (cf. Tab. \ref{['table:vibration_sensing']}). In Fig. \ref{['fig:force_reconstruct']}, Evetac presses against an object mounted on top of a F/T sensor. Through moving the robot, we shear the gel. By combining our proposed dot tracking algorithm with a model, we attempt to recover the shear forces (cf. Fig. \ref{['fig:force_recon']}).