An Event-Based Opto-Tactile Skin

TL;DR

This work introduces a neuromorphic, event-driven tactile sensing skin that uses stereo DVS cameras looking through a flexible silicone layer to localize contact points via triangulation. By clustering surface events with DBSCAN and restricting localization to the horizontal image coordinates, the system achieves a RMSE of $4.66\ \mathrm{mm}$ over a large probed area, with most probes visible to both cameras. Significantly, the approach maintains a pass-rate above $85\%$ even when the event stream is reduced by up to $1024\times$, at a latency around $31\ \mathrm{ms}$, indicating strong potential for low-bandwidth, real-time tactile sensing in soft robotics. The marker-free design avoids embedded electronics in the sensing surface and scales to large areas, enabling robust interaction in dynamic environments, while also outlining concrete paths to handle multi-touch scenarios and further sensitivity improvements in future work.

Abstract

This paper presents a neuromorphic, event-driven tactile sensing system for soft, large-area skin, based on the Dynamic Vision Sensors (DVS) integrated with a flexible silicone optical waveguide skin. Instead of repetitively scanning embedded photoreceivers, this design uses a stereo vision setup comprising two DVS cameras looking sideways through the skin. Such a design produces events as changes in brightness are detected, and estimates press positions on the 2D skin surface through triangulation, utilizing Density-Based Spatial Clustering of Applications with Noise (DBSCAN) to find the center of mass of contact events resulting from pressing actions. The system is evaluated over a 4620 mm2 probed area of the skin using a meander raster scan. Across 95 % of the presses visible to both cameras, the press localization achieved a Root-Mean-Squared Error (RMSE) of 4.66 mm. The results highlight the potential of this approach for wide-area flexible and responsive tactile sensors in soft robotics and interactive environments. Moreover, we examined how the system performs when the amount of event data is strongly reduced. Using stochastic down-sampling, the event stream was reduced to 1/1024 of its original size. Under this extreme reduction, the average localization error increased only slightly (from 4.66 mm to 9.33 mm), and the system still produced valid press localizations for 85 % of the trials. This reduction in pass rate is expected, as some presses no longer produce enough events to form a reliable cluster for triangulation. These results show that the sensing approach remains functional even with very sparse event data, which is promising for reducing power consumption and computational load in future implementations. The system exhibits a detection latency distribution with a characteristic width of 31 ms.

Figures (9)

• Figure 1: Exploded view of the sensor design. Components: (1) ABS printed base; (2) emitter mounts; (3) lens mounts; (4) near-infrared VSMY1860 emitters; (5) ABS lens replicas used during molding; (6) PDMS tactile layer.
• Figure 2: Experimental setup for tactile data acquisition with our optical skin system. The setup includes DVS cameras (2 and 3) placed on two adjacent sides of a rectangular silicone layer to capture dynamic touch events. The Omega 3 force dimension robot (1) applies a controlled force using the tip (5) to predefined points on the silicone surface, creating deformations detected by the cameras. NIR LEDs (4) provide consistent illumination, allowing detection of surface deformations.
• Figure 3: Overview of DVS camera event data illustrating press actions localization based on the circular path. Image A shows the accumulated event rates across all contacts, providing a high-level view of spatial activity distribution over the sensing area. The yellow box indicates the cropped region (v = 200--360), and only the pixels inside this box are retained for analysis. Images B, C, and D represent individual contacts captured by the same camera, each shown in a color-coded channel. These images demonstrate the sensor's sensitivity to different press locations, as each press generates a unique pattern of event distribution. Images E, F, G, and H (right column) display a temporal sequence for a single contact, highlighting spatiotemporal characteristics that could be useful for further classification.
• Figure 4: Event rate analysis during skin pressing, computed using histograms with a bin size of 10 . (A) Event rate over the first 100 of activity from Camera 1 (red) and Camera 2 (blue), corresponding to the first 27 presses performed in a meandering path. The inset shows the sensor area and pressing path, with camera positions indicated (red and blue) and individual presses marked. Selected presses are highlighted with color-coded rectangles on the event-rate plot and linked to circles of the same color on the sensor map: green for the 2nd press, yellow for the 16th press, and purple for the 27th press. (B) Event-rate for a single pressure instance recorded by both cameras, overlaid to show synchronization and comparative sensitivity. Camera 1 is plotted in red and Camera 2 in blue consistently.
• Figure 5: The spatial distribution of pixel activity using DBSCAN. This figure visualizes the estimated activity centers across the sensor surface based on DBSCAN clustering of pixel events. (A) and (B) show the top-down views of the $u-$coordinate centroids extracted from each press for Camera 1 and Camera 2, respectively. These views illustrate the spatial spread of detected presses along the circular path. (C) and (D) show example contact activity for Camera 1 and Camera 2, where the DBSCAN-identified pixel clusters have been processed to extract their centroids, representing the dominant activity locations. In these examples, the clusters correspond to press number 95, which is marked with a black circle in subplots A and B, and the black cross in subplots C and D indicates the center of the activity for that press.