LightTact: A Visual-Tactile Fingertip Sensor for Deformation-Independent Contact Sensing

TL;DR

LightTact addresses the challenge of sensing contact with very light touches that produce negligible surface deformation by introducing an ambient-blocking, deformation-independent optical layout. A nonparallel wedge between the touching and viewing surfaces suppresses non-contact light while transmitting only contact-generated diffuse light, yielding high-contrast, appearance-preserving tactile images that enable robust pixel-level contact segmentation. The sensor demonstrates reliable operation across liquids, ultra-soft materials, and rigid objects, and, when mounted on a robotic arm, enables light-contact manipulation tasks such as water spreading and facial-cream dipping, as well as multimodal reasoning by prompting LightTact images to vision-language models for tasks like resistor sorting. This approach offers a practical, VLM-friendly form of tactile perception that remains robust to lighting and material appearance, promising new capabilities for contact-rich manipulation in unstructured environments.

Abstract

Contact often occurs without macroscopic surface deformation, such as during interaction with liquids, semi-liquids, or ultra-soft materials. Most existing tactile sensors rely on deformation to infer contact, making such light-contact interactions difficult to perceive robustly. To address this, we present LightTact, a visual-tactile fingertip sensor that makes contact directly visible via a deformation-independent, optics-based principle. LightTact uses an ambient-blocking optical configuration that suppresses both external light and internal illumination at non-contact regions, while transmitting only the diffuse light generated at true contacts. As a result, LightTact produces high-contrast raw images in which non-contact pixels remain near-black (mean gray value < 3) and contact pixels preserve the natural appearance of the contacting surface. Built on this, LightTact achieves accurate pixel-level contact segmentation that is robust to material properties, contact force, surface appearance, and environmental lighting. We further integrate LightTact on a robotic arm and demonstrate manipulation behaviors driven by extremely light contact, including water spreading, facial-cream dipping, and thin-film interaction. Finally, we show that LightTact's spatially aligned visual-tactile images can be directly interpreted by existing vision-language models, enabling resistor value reasoning for robotic sorting.

Figures (14)

• Figure 1: LightTact provides direct, pixel-level contact sensing across liquids, ultra-soft materials, and rigid objects, without requiring a minimum contact force. Its optical design produces high-contrast raw images in which non-contact pixels remain near-black, while contact pixels preserve the natural appearance of the contacting surface. LightTact is fingertip-sized and can be integrated into dexterous hands such as Amazing Hand pollenrobotics_amazinghand_github_2025.
• Figure 2: Optical layout and sensing principle of LightTact. (a) Core components: a transparent medium with a touching and viewing surface, an internal LED, and a camera. (b) External light entering through non-contact regions is rejected by refraction followed by total internal reflection (TIR). (c) Internal LED illumination reflecting from non-contact regions is also rejected by TIR when the LED is placed within the purple area to satisfy $\theta_{it}<\theta_i$. (d) At true contact, the air gap is removed and the contacting surface produces diffuse reflection. A subset of these rays reaches the viewing surface with incidence $<\theta_c$ and refracts toward the camera.
• Figure 3: Design of LightTact. (a) Assembled view of LightTact. (b) Components of LightTact illustrated in exploded view. (c) Schematic diagram of LightTact.
• Figure 4: Gel fabrication for LightTact. (a) Casting of the transparent gel. The sensor shell, the mold 0, and the acrylic window together form a closed container. (b) Casting of the black gel. The top acrylic mold 4 ensures that the black gel cures to a uniform height around the transparent gel.
• Figure 5: Camera calibration for LightTact. (a) LightTact captures imprints produced by a calibration tool with a $5\times5$ array of cylinders. (b) The imprints are segmented by thresholding and their center pixels are detected. (c) The final rectified and cropped image.