Grasping, Part Identification, and Pose Refinement in One Shot with a Tactile Gripper

TL;DR

The paper addresses the challenge of automating post-processing for massively customized 3D-printed parts by introducing pattern augmentation on the parts themselves. A tactile-imprinting approach with a GelSight sensor enables one-shot grasping, part identification, and pose refinement by matching tactile imprints against a pattern library created via simulated annealing on a Delaunay grid, followed by voxelized point-cloud registration for sub-millimeter refinement; this yields a $95\%$ insertion success rate and $<$1 mm refinement in real tasks, with identification and refinement occurring in about $0.4\,\text{s}$. Key contributions include (i) a scalable pattern-library generation method producing 1095 patterns, (ii) an IoU-based imprint classification coupled with pattern-driven pose refinement, and (iii) demonstrated end-to-end robotic sorting and packing that outperforms vision-only approaches in object differentiation for customized parts. The work has practical impact by enabling fast, end-to-end automation of 3D-printed post-processing, reducing manual labor and accelerating industrial workflows, while also highlighting areas for standalone deployment and improved pattern-distribution strategies in future work.

Abstract

The rise in additive manufacturing comes with unique opportunities and challenges. Rapid changes to part design and massive part customization distinctive to 3D-Print (3DP) can be easily achieved. Customized parts that are unique, yet exhibit similar features such as dental moulds, shoe insoles, or engine vanes could be industrially manufactured with 3DP. However, the opportunity for massive part customization comes with unique challenges for the existing production paradigm of robotics applications, as the current robotics paradigm for part identification and pose refinement is repetitive, where data-driven and object-dependent approaches are often used. Thus, a bottleneck exists in robotics applications for 3DP parts where massive customization is involved, as it is difficult for feature-based deep learning approaches to distinguish between similar parts such as shoe insoles belonging to different people. As such, we propose a method that augments patterns on 3DP parts so that grasping, part identification, and pose refinement can be executed in one shot with a tactile gripper. We also experimentally evaluate our approach from three perspectives, including real insertion tasks that mimic robotic sorting and packing, and achieved excellent classification results, a high insertion success rate of 95%, and a sub-millimeter pose refinement accuracy.

Figures (5)

• Figure 1: Pattern augmentation on 3DP parts for object recognition and high accuracy pose refinement to conduct insertion tasks. A video demonstration is available at https://youtu.be/3e6gvkZUk8c
• Figure 2: Graphical pipeline for object classification and pose refinement for pattern augmented 3DP objects.
• Figure 3: A unique pattern library is obtained by using simulated annealing to place triangles on a grid. The pattern library and the STL files of the objects are used to create pattern-augmented objects and their corresponding labels. Labels correspond to patterns rather than objects.
• Figure 4: Random initial pose of robot manipulator: (a) Illustration of perturbation axes; (b) Cube initial position is unknown after grasping which resulted from the random perturbation of robot manipulator.
• Figure 5: Robotic sorting and packing into shadow boxes. Three objects were shown in the video (https://youtu.be/3e6gvkZUk8c) and the dimensional allowance between the objects and holes ranges from 1.3mm to 2.3mm.