RainbowSight: A Family of Generalizable, Curved, Camera-Based Tactile Sensors For Shape Reconstruction

TL;DR

RainbowSight, a family of curved, compact, camera-based tactile sensors which use addressable RGB LEDs illuminated in a novel rainbow spectrum pattern to make the integration of tactile sensors more accessible to roboticists by allowing them the flexibility to easily customize, fabricate, and calibrate camera-based tactile sensors to best fit the needs of their robotic systems.

Abstract

Camera-based tactile sensors can provide high resolution positional and local geometry information for robotic manipulation. Curved and rounded fingers are often advantageous, but it can be difficult to derive illumination systems that work well within curved geometries. To address this issue, we introduce RainbowSight, a family of curved, compact, camera-based tactile sensors which use addressable RGB LEDs illuminated in a novel rainbow spectrum pattern. In addition to being able to scale the illumination scheme to different sensor sizes and shapes to fit on a variety of end effector configurations, the sensors can be easily manufactured and require minimal optical tuning to obtain high resolution depth reconstructions of an object deforming the sensor's soft elastomer surface. Additionally, we show the advantages of our new hardware design and improvements in calibration methods for accurate depth map generation when compared to alternative lighting methods commonly implemented in previous camera-based tactile sensors. With these advancements, we make the integration of tactile sensors more accessible to roboticists by allowing them the flexibility to easily customize, fabricate, and calibrate camera-based tactile sensors to best fit the needs of their robotic systems.

Figures (7)

• Figure 1: Two omnidirectional RainbowSight fingers mounted on a parallel-jaw gripper holding an M7 screw. (A) Rainbow illuminated addressable RGB LED ring mounted at the base of the sensors. To the right is an example of a scaled-down version of the sensor with a diameter similar to that of a dime ($\sim$20 mm). (B) Example difference image captured by camera showing the deformation in the elastomer coating. (C) Depth reconstruction of the sensor surface viewed from sensor base. (D) Point cloud of sensor surface viewed from camera base.
• Figure 2: Exploded view of sensor.
• Figure 3: Example difference images collected when a 4mm ball and M7 screw are pressed into the various sensor shapes and sizes. In the half-sensor configurations and tops of omnidirectional sensors, we are able to achieve the desired rainbow gradient illumination pattern over a large area of the main sensing surface. For omnidirectional sensors, the sensor sides provide adjacent two-color parallel light.
• Figure 4: Top Dashed Box: Depth reconstruction pipeline. Bottom: Tactile signals collected when different objects are pressed at various locations on the omnidirectional cylinder with hemisphere top sensor shape. Top Row: Objects pressed into the sensor surface. Middle Row: Tactile difference images of the contact regions. Bottom Row: Estimated depth map of the imprinted object in the sensor skin.
• Figure 5: Comparison of the color gradients seen when illuminating a curved sensor shape that is not radially symmetric around the vertical axis. The lack of blending colors can be seen on the left when only red, green, and blue light illuminates the sensor. A more gradual gradation of colors can be seen in the rainbow illuminated sensor.