Object Recognition, Dynamic Contact Simulation, Detection, and Control of the Flexible Musculoskeletal Hand Using a Recurrent Neural Network with Parametric Bias

TL;DR

This work tackles the difficulty of modeling a flexible musculoskeletal hand whose dynamics drift over time and initialization differences by introducing HADYNET, a recurrent neural network with parametric bias that encodes multiple hand dynamics into a compact PB. By learning a sensor-state equation $\bm{z}_{t}= \bm{h}_{1}(\bm{x}_{t}, \bm{p})$ and $\bm{y}_{t+1}= \bm{h}_{2}(\bm{z}_{t}, \bm{u}_{t}, \bm{p})$, the approach enables simultaneous object recognition, contact simulation, detection, and control within a single integrated network, adapting online by updating $\bm{p}$. The key contributions include (1) acquiring a PB-embedded sensor state model for a flexible hand, (2) demonstrating PB-based recognition of grasped objects and online adaptation to deterioration or initialization changes, (3) enabling contact simulation, anomaly detection via Mahalanobis distance, and gradient-based contact control, all implemented on the Musashi hand. The results show that updating only PB suffices to capture different grasp dynamics, supporting stable control and rapid adaptation, with practical impact for robust manipulation using soft or flexible hands in changing environments.

Abstract

The flexible musculoskeletal hand is difficult to modelize, and its model can change constantly due to deterioration over time, irreproducibility of initialization, etc. Also, for object recognition, contact detection, and contact control using the hand, it is desirable not to use a neural network trained for each task, but to use only one integrated network. Therefore, we develop a method to acquire a sensor state equation of the musculoskeletal hand using a recurrent neural network with parametric bias. By using this network, the hand can realize recognition of the grasped object, contact simulation, detection, and control, and can cope with deterioration over time, irreproducibility of initialization, etc. by updating parametric bias. We apply this study to the hand of the musculoskeletal humanoid Musashi and show its effectiveness.

Figures (15)

• Figure 1: Overview of the developed system.
• Figure 2: The five-finger flexible musculoskeletal hand makino2018hand of the musculoskeletal humanoid Musashi kawaharazuka2019musashi.
• Figure 3: Network structure of hand dynamics network (HADYNET).
• Figure 4: The overall system using the hand dynamics network.
• Figure 5: Grasped objects in this study: Hammer, Hammer-S (hammer with soft cover), Cylinder, Gripper, and Ball.