Enhance Vision-based Tactile Sensors via Dynamic Illumination and Image Fusion

TL;DR

Vision-based tactile sensors traditionally rely on static illumination, which limits sensing fidelity. The authors propose dynamic illumination with multi-pattern acquisitions and image fusion, notably Discrete Wavelet Transform fusion, to produce higher-quality tactile images. Their framework defines Θ, n, and f and uses multiple metrics to optimize image quality, demonstrating across eight objects that dynamic lighting improves contrast, sharpness, and background differentiation, with DWT fusion achieving the best balance across metrics. They analyze optimal sequence lengths and timing, finding that typically 2–4 images maximize sharpness while 1 image often suffices for contrast, and that a ~0.29s frame interval yields stable improvements. This approach offers a software-update path to retroactively improve existing VBTS sensors and informs hardware designs to exploit dynamic illumination.

Abstract

Vision-based tactile sensors use structured light to measure deformation in their elastomeric interface. Until now, vision-based tactile sensors such as DIGIT and GelSight have been using a single, static pattern of structured light tuned to the specific form factor of the sensor. In this work, we investigate the effectiveness of dynamic illumination patterns, in conjunction with image fusion techniques, to improve the quality of sensing of vision-based tactile sensors. Specifically, we propose to capture multiple measurements, each with a different illumination pattern, and then fuse them together to obtain a single, higher-quality measurement. Experimental results demonstrate that this type of dynamic illumination yields significant improvements in image contrast, sharpness, and background difference. This discovery opens the possibility of retroactively improving the sensing quality of existing vision-based tactile sensors with a simple software update, and for new hardware designs capable of fully exploiting dynamic illumination.

Figures (8)

• Figure 1: Current vision-based tactile sensors use static illumination patterns. In this work, we instead propose to collect several measurements under dynamic illumination conditions, and then fuse them together in a single higher-quality measurement. Experimental results show that this approach yields significantly improved quality of sensing.
• Figure 2: Images of a coin, and corresponding measurements obtained with DIGIT with different illumination settings.
• Figure 3: Objects used in the experiments.
• Figure 4: Heatmaps showing changes in contrast and sharpness of the image that is a result of image fusion of image taken with standard illumination and one more image taken with different illumination. The greatest contrast increase was obtained when adding the image obtained with only green and blue LED lights on (0,10,3) and the greatest increase of sharpness was obtained setting intensities of RGB lights to (0,10,3).
• Figure 5: Measurements of a coin and Lego brick obtained using dynamic illumination and various image fusion methods