A multi-modal tactile fingertip design for robotic hands to enhance dexterous manipulation

TL;DR

This work introduces a compact, low-cost multi-modal tactile fingertip that embeds strain gauges for static force sensing and a contact microphone for vibrotactile sensing inside the fingertip. The design emphasizes manufacturability, durability, and ease of integration, achieving a footprint of $1.9\,\mathrm{cm}\times1.9\,\mathrm{cm}\times2.7\,\mathrm{cm}$ and a cost below $100. Through extensive characterization, the fingertip provides accurate 2D force estimates in the $0$–$5$ N range and high material-discrimination accuracy ($\approx$ $95.49\%$) from vibrotactile signals. Evaluations on three manipulation tasks show that tactile sensing can either supplement or replace vision under occlusion, delivering perfect performance on fragile-object pinching and cup counting, and strong performance in hidden-object material detection ($94.64\%$), highlighting the practical impact of compact, interpretable tactile sensing for dexterous robotic manipulation.

Abstract

Tactile sensing holds great promise for enhancing manipulation precision and versatility, but its adoption in robotic hands remains limited due to high sensor costs, manufacturing and integration challenges, and difficulties in extracting expressive and reliable information from signals. In this work, we present a low-cost, easy-to-make, adaptable, and compact fingertip design for robotic hands that integrates multi-modal tactile sensors. We use strain gauge sensors to capture static forces and a contact microphone sensor to measure high-frequency vibrations during contact. These tactile sensors are integrated into a compact design with a minimal sensor footprint, and all sensors are internal to the fingertip and therefore not susceptible to direct wear and tear from interactions. From sensor characterization, we show that strain gauge sensors provide repeatable 2D planar force measurements in the 0-5 N range and the contact microphone sensor has the capability to distinguish contact material properties. We apply our design to three dexterous manipulation tasks that range from zero to full visual occlusion. Given the expressiveness and reliability of tactile sensor readings, we show that different tactile sensing modalities can be used flexibly in different stages of manipulation, solely or together with visual observations to achieve improved task performance. For instance, we can precisely count and unstack a desired number of paper cups from a stack with 100\% success rate which is hard to achieve with vision only.

Figures (7)

• Figure 1: (a) Fingertip structure includes a rigid bone and a fingernail, and is covered by a soft skin, which mimics human fingertip's mechanical structure. We leverage strain gauge sensors to emulate the role of slow adapting (SA) mechanoreceptors, responsible for detecting pressure, and a contact microphone sensor to replicate the function of fast adapting (FA) mechanoreceptors, to pick up contact vibrations and infer dynamic contact events such as slip kandel2000principles. (b) A Delta finger equipped with the proposed fingertip. (c) A DeltaHand 10380685 equipped with four tactile fingertips. (d) (e) Grasp a soft tofu cube and a single potato chip without damaging by using tactile sensor measurements in the hand control loop.
• Figure 2: The fabrication procedure and final prototype of the fingertip. (a) Fingertip fabrication: $\textcircled{1}$ Soldering the connector onto the PCB; $\textcircled{2}$ attaching the contact microphone sensor on the top of the 3D printed cap using super glue; $\textcircled{3}$ molding a soft skin for the cap with silicone (Mold Star 20T) by using two 3D-printed molds; $\textcircled{4}$ gluing four strain gauge sensors on each side of the square prism; $\textcircled{5}$ mounting the PCB with M2 screws below the fingertip base; $\textcircled{6}$ Assembling the cap and the base with an M2 screw; $\textcircled{7}$ wearing the soft skin on the cap. (b) A fully assembled fingertip prototype.
• Figure 3: Readout circuit of the fingertip: the contact microphone sensor signals and the strain gauge sensor signals are collected through the fingertip PCB and carried by an FFC cable to another custom PCB. The strain gauge sensor measurements are amplified and digitized with an HX711 module on an Arduino Uno, while the vibrotactile signals are pre-amplified, digitized by a modified Maono USB sound card. Both are transmitted to the PC via USB.
• Figure 4: Tactile sensor characterization setup and results: (a) Front and side views of the setup for strain gauge sensor characterization. A fingertip sits on a 6D force/torque sensor that is rigidly fixed on the table. A UR5e robotic arm with a custom end effector ($3$ mm cylindrical indenter) is used to step and press on the fingertip from different directions and with incremental indentation depths to characterize 2D planar force sensing. (b) Front and side views of the setup for contact microphone sensor characterization. We use a custom end-effector with samples of seven materials attached on different faces. The UR5e arm is controlled to initiate sliding contacts between the fingernail and a sample to generate vibrotactile signals. (c) We show the linear correspondence between the strain gauge sensor measurements and the ground-truth force readings from $0$ to $5$ N at a fixed indentation depth across different directions. (d) We learn a material classification model by using vibrotactile measurements. We show the results on the test dataset.
• Figure 5: Pinching and lifting fragile objects including tofu cubes and potato chips with force control based on fingertip strain gauge sensors. We set a force threshold (0.5 N for tofu cubes, and 0.1 N for potato chips) to guide pinching, and maintain the force applied on the object during lifting.