MagicSkin: Balancing Marker and Markerless Modes in Vision-Based Tactile Sensors with a Translucent Skin

TL;DR

<p>Vision-based tactile sensors face an inherent trade-off between marker-based tangential tracking and preserving surface details for texture/object recognition. MagicSkin introduces translucent, tinted markers embedded in the elastomer to balance both modalities without extra hardware or software, enabling simultaneous displacement tracking, force prediction, and surface detail preservation. Across object and texture classification, tangential tracking, and 3-axis force prediction, MagicSkin with grey-square markers achieves top performance (object 99.17%, texture 93.51%, tangential tracking 0.014 px FB error with 97% retention, force MAE improvement >66%). This approach eliminates the traditional trade-offs of marker-based and markerless designs, offering a practical, scalable route toward multimodal tactile sensing for manipulation and dexterous grasping.</p>

Abstract

Vision-based tactile sensors (VBTS) face a fundamental trade-off in marker and markerless design on the tactile skin: opaque ink markers enable measurement of force and tangential displacement but completely occlude geometric features necessary for object and texture classification, while markerless skin preserves surface details but struggles in measuring tangential displacements effectively. Current practice to solve the above problem via UV lighting or virtual transfer using learning-based models introduces hardware complexity or computing burdens. This paper introduces MagicSkin, a novel tactile skin with translucent, tinted markers balancing the modes of marker and markerless for VBTS. It enables simultaneous tangential displacement tracking, force prediction, and surface detail preservation. This skin is easy to plug into GelSight-family sensors without requiring additional hardware or software tools. We comprehensively evaluate MagicSkin in downstream tasks. The translucent markers impressively enhance rather than degrade sensing performance compared with traditional markerless and inked marker design: it achieves best performance in object classification (99.17\%), texture classification (93.51\%), tangential displacement tracking (97\% point retention) and force prediction (66\% improvement in total force error). These experimental results demonstrate that translucent skin eliminates the traditional performance trade-off in marker or markerless modes, paving the way for multimodal tactile sensing essential in tactile robotics. See videos at this \href{https://zhuochenn.github.io/MagicSkin_project/}{link}.

Figures (10)

• Figure 1: Demonstration of MagicSkin. Current practices use dense ink design and clear skin for marker and markerless modes in VBTS respectively. Our Magic skin can balance above two modes in one translucent skin for marker tracking, force prediction and object/texture classification without introducing extra hardware or software to switch.
• Figure 2: Fabrication process (a) and two designed translucent skins (b).
• Figure 3: Sensor assembly.
• Figure 4: Objects used in object classification (a) and texture recognition (b)
• Figure 5: Comparison of tactile image for object classification (a) and texture recognition (b) using different tactile skin design.