AllTact Fin Ray: A Compliant Robot Gripper with Omni-Directional Tactile Sensing

TL;DR

AllTact Fin Ray tackles the challenge of safe, precise manipulation with soft grippers by delivering a compliant two‑finger gripper with omni‑directional tactile sensing. The core method reconstructs global finger deformation from a single image using the pinhole model $z \mathbf{p}' = \mathbf{K} \mathbf{p}$ together with a width constraint on edges, and extracts local contact geometry via brightness changes against a dynamically retrieved reference image using the mapping $M(\widetilde{ \Delta I}) = d$ with $\widetilde{ \Delta I} = (I_{ref}-I)/I_{ref}$. A YOLO‑based marker detector and dynamic reference video enable real‑time omni‑directional contact detection and force estimation, achieving per‑frame processing of about 18 ms. Experimental results show sub‑millimeter contact localization in the central region, accurate object pose estimation and reliable grasping and pose adjustment across varied shapes and lighting, demonstrating practical applicability for manipulation. The work also provides openly accessible design files and sensing algorithms to accelerate deployment and further research.

Abstract

Tactile sensing plays a crucial role in robot grasping and manipulation by providing essential contact information between the robot and the environment. In this paper, we present AllTact Fin Ray, a novel compliant gripper design with omni-directional and local tactile sensing capabilities. The finger body is unibody-casted using transparent elastic silicone, and a camera positioned at the base of the finger captures the deformation of the whole body and the contact face. Due to the global deformation of the adaptive structure, existing vision-based tactile sensing approaches that assume constant illumination are no longer applicable. To address this, we propose a novel sensing method where the global deformation is first reconstructed from the image using edge features and spatial constraints. Then, detailed contact geometry is computed from the brightness difference against a dynamically retrieved reference image. Extensive experiments validate the effectiveness of our proposed gripper design and sensing method in contact detection, force estimation, object grasping, and precise manipulation.

Figures (21)

• Figure 1: AllTact Fin Ray finger structure view. (a) Simplified cutaway view where the Fin Ray finger body is not cut. (b) Complete exploded view.
• Figure 2: Diagram of the Fabrication Process for AllTact Fin Ray Finger
• Figure 3: Full view of AllTact Fin Ray gripper. (a) View of physical gripper. (b) Cutaway view of gripper in CAD model.
• Figure 4: Diagram of deformation and calibraion of our finger during contact, where a ball with known radius is pressed onto the contact face for sensor calibration. The camera pinhole model is employed to compute the 3D coordinate of ${\bf{c}}$, which is further used to calculate the press depth.
• Figure 5: Diagram of global deformation reconstruction. We use $W$ and the image coordinates ${{\bf{p}}'_1}$ and ${{\bf{p}}'_2}$ on the two edges to compute the depth and 3D coordinates of points. Note that ${{\bf{p}}_1}$ and ${{\bf{p}}_2}$ are not markers doted on side faces but drawn on the two edges of the contact face.