SymmeTac: Symmetric Color LED Driven Efficient Photometric Stereo Reconstruction Methods for Camera-based Tactile Sensors

TL;DR

This work designs orthogonal red and blue LEDs as illumination to acquire four observation maps using channel-splitting in a two-shot manner, and develops a two-shot photometric stereo theory, which can estimate accurate surface normal and greatly reduce the computing overhead in magnitude.

Abstract

Camera-based tactile sensors can provide high-density surface geometry and force information for robots in the interaction process with the target. However, most existing methods cannot achieve accurate reconstruction with high efficiency, impeding the applications in robots. To address these problems, we propose an efficient two-shot photometric stereo method based on symmetric color LED distribution. Specifically, based on the sensing response curve of CMOS channels, we design orthogonal red and blue LEDs as illumination to acquire four observation maps using channel-splitting in a two-shot manner. Subsequently, we develop a two-shot photometric stereo theory, which can estimate accurate surface normal and greatly reduce the computing overhead in magnitude. Finally, leveraging the characteristics of the camera-based tactile sensor, we optimize the algorithm to be a highly efficient, pure addition operation. Simulation and real-world experiments demonstrate the advantages of our approach. Further details are available on: https://github.com/Tacxels/SymmeTac.

Figures (12)

• Figure 1: (a) Camera-based tactile sensor with symmetric illuminations. (b) Camera response curve supports channel splitting. (c) Proposed two-shot, highly efficient PS method and the reconstructed surface normal map.
• Figure 2: Proposed two-shot photometric stereo with two-shot illuminations. By utilizing the spectral characteristics of chromatic CMOS (red and blue channel has no (little) crosstalk) and LEDs in (a) and their predefined position in (b), red and blue channels can be split from image as (c).
• Figure 3: The comparison of computational complexity between typical algorithms for camera-based tactile sensor and our methods (ST-A, ST-B).
• Figure 4: Reconstruction results from typical photometric stereo algorithms and the proposed method. We choose five relief objects to represent the challenging tactile surface (1-7 cols). We also introduce spatial-varying and channel-wise albedo in the rendering process. The robustness under shadows and specular surface is evaluated (8-11 cols). The mean angular error is attached on the left corner of each error map (color range is [0$^\circ$, 10$^\circ$]).
• Figure 5: Results from our two-shot PS and classical LSPS method under both directional illumination and near-light illumination. The normal map show slight bulge caused by near-light nonuniform and non-parallel illumination.