Using Fiber Optic Bundles to Miniaturize Vision-Based Tactile Sensors

TL;DR

This work presents DIGIT Pinki, a 15 mm diameter, vision-based tactile sensor that uses optical fiber bundles to decouple the sensing elastomer from proximal electronics, achieving a spatial resolution of 0.22 mm and a normal/shear force resolution of ~5 mN. The distal elastomer gel is illuminated and imaged through fiber bundles, enabling a human-scale fingertip with remote imaging and lighting, and a modular screw-on gel tip for easy swapping. Extensive experiments show high-resolution tactile imaging, accurate image-to-force regression (normal and shear), and strong hardness discrimination across silicone phantoms and phantom/ex vivo prostate tissues, including a phantom study with 100% classification accuracy and an ex vivo case with perfect test accuracy. The work also analyzes contact mechanics, sequence-length effects on hardness classification, and outlines future design directions (e.g., hyperfisheye optics, higher core counts) and potential medical applications, with the design openly available as open-source hardware resources.

Abstract

Vision-based tactile sensors have recently become popular due to their combination of low cost, very high spatial resolution, and ease of integration using widely available miniature cameras. The associated field of view and focal length, however, are difficult to package in a human-sized finger. In this paper we employ optical fiber bundles to achieve a form factor that, at 15 mm diameter, is smaller than an average human fingertip. The electronics and camera are also located remotely, further reducing package size. The sensor achieves a spatial resolution of 0.22 mm and a minimum force resolution 5 mN for normal and shear contact forces. With these attributes, the DIGIT Pinki sensor is suitable for applications such as robotic and teleoperated digital palpation. We demonstrate its utility for palpation of the prostate gland and show that it can achieve clinically relevant discrimination of prostate stiffness for phantom and ex vivo tissue.

Figures (20)

• Figure 1: (a) DIGIT lambeta2020digit(left) and DIGIT 360 lambeta2024digit360(middle) compared with the DIGIT Pinki (right) introduced in this work. The DIGIT Pinki's diameter of 15mm is achieved using optical fiber bundles as illumination and imaging conduits, thereby allowing the elastomer sensing element to achieve human scale. (b) DIGIT Pinki's off-the-shelf camera and supporting circuitry would be difficult to package in the base of a typical vision-based tactile sensor for the same tip size. (c) An image from DIGIT Pinki pressed against the "ONE" text of a U.S. dime, with its sub-millimeter structure visible.
• Figure 2: The proposed system consists of: a distal sensing element, a proximal imaging system, and a proximal illumination system, all of which are connected with optical fiber bundles. The distal end contains (a) an elastomer gel mounted in (b) a 3D-printed housing (pictured at left is a housing compatible with the Allegro Hand, but this housing could be designed for other uses). Both the (c) imaging and (d) illumination fiber bundles mate to the gel. At the proximal end, the illumination fiber bundles mate to the (e) collimator coupled to (f) LEDs. When making contact with an object (e.g. a woman's index finger), the gel is first imaged with (g) a distal lens, conveyed through the imaging fiber bundle, then magnified by (h) optical lenses and captured by the (i) camera. The proximal imaging and illumination systems may be co-located or separate.
• Figure 3: Comparison of the ray-traced outputs in Zemax Optics Studio of lighting from a 5050 RGB LED package (\ref{['fig:illuminationWithoutFiber']}) without and (\ref{['fig:illuminationWithFiber']}) with the fiber bundle. The fiber output is narrower and more focused.
• Figure 4: (a) The USAF 1951 Test Pattern resolution target as captured with the imaging fiber (no gel). The working distance is set to the height of the gel. Group 1, element 2, which is outlined in white, can still be resolved (lines do not blur). When zoomed in to the output image, there is an apparent "honeycomb" pattern, as is typical when imaging through fibers. (b-e) Readings from a DIGIT Pinki when pressed against machined calibration blocks with known line widths of: (b) 1mm, (c) 0.50mm, (d) 0.35mm, and (e) 0.25mm. The indentations of the lines are visible.
• Figure 5: (\ref{['fig:characterizationSetup']}) Benchmarking setup used to collect force data over three different indenter types. From the dataset, we train the image-to-force prediction models. (\ref{['fig:normalPrediction']}) We plot the predicted force compared to ground truth for one indentation in the normal force dataset from 0N to -1N. (\ref{['fig:shearPrediction']}) We plot the predicted forces compared to the ground truth for one indentation from the shear force dataset. The robot indents -0.5N in the normal direction (green) before loading in shear (orange).