Soft and Rigid Object Grasping With Cross-Structure Hand Using Bilateral Control-Based Imitation Learning
Koki Yamane, Sho Sakaino, Toshiaki Tsuji
TL;DR
This work tackles object grasping requiring precise force adjustments by introducing a cross-structure rigid hand paired with bilateral control-based imitation learning. It leverages a 4-channel bilateral control setup to collect force-augmented demonstrations and trains an F2FL LSTM policy to predict next-step leader and follower commands, enabling fast, human-like manipulation. The method demonstrates high success in both soft grasping (pick-and-place with varied objects) and rigid grasping (letter writing with a tool), highlighting the practical potential of force-aware imitation learning with a simple hand. The approach offers a data-efficient path to robust grasping across object varieties and tool use, with implications for rapid deployment in real-world robotics tasks.
Abstract
Object grasping is an important ability required for various robot tasks. In particular, tasks that require precise force adjustments during operation, such as grasping an unknown object or using a grasped tool, are difficult for humans to program in advance. Recently, AI-based algorithms that can imitate human force skills have been actively explored as a solution. In particular, bilateral control-based imitation learning achieves human-level motion speeds with environmental adaptability, only requiring human demonstration and without programming. However, owing to hardware limitations, its grasping performance remains limited, and tasks that involves grasping various objects are yet to be achieved. Here, we developed a cross-structure hand to grasp various objects. We experimentally demonstrated that the integration of bilateral control-based imitation learning and the cross-structure hand is effective for grasping various objects and harnessing tools.