Adaptive Robotic Tool-Tip Control Learning Considering Online Changes in Grasping State

TL;DR

This work tackles tool-tip control under online changes in grasping state and deformable tools by introducing the Tool-Body Network with Parametric Bias (TBNPB), which models the static relation $\bm{x}_{tool} = \bm{h}(\bm{u}, \bm{p})$ while encoding the grasping state in the parametric bias $\bm{p}$. The network is trained in two stages (offline simulation then real-robot fine-tuning), with online updates of $\bm{p}$ enabling adaptation to changing grasping states without retraining the full network. Experiments on PR2 (rigid) and MusashiLarm (flexible) demonstrate improved tool-tip estimation and control, including scenarios with deformable tools and abrupt grasping-state changes. While effective, the approach faces data scalability and tool-type coverage challenges, and future work includes dynamic tool-use and sensor fusion integration to broaden applicability.

Abstract

Various robotic tool manipulation methods have been developed so far. However, to our knowledge, none of them have taken into account the fact that the grasping state such as grasping position and tool angle can change at any time during the tool manipulation. In addition, there are few studies that can handle deformable tools. In this study, we develop a method for estimating the position of a tool-tip, controlling the tool-tip, and handling online adaptation to changes in the relationship between the body and the tool, using a neural network including parametric bias. We demonstrate the effectiveness of our method for online change in grasping state and for deformable tools, in experiments using two different types of robots: axis-driven robot PR2 and tendon-driven robot MusashiLarm.

Figures (16)

• Figure 1: The concept of this study. In robotic tool-use, a tool-tip posture is estimated from the body control command and grasping state, the body control command is calculated from the loss between the target and estimated tool-tip postures, and grasping state is updated online from the loss between the estimated and measured tool-tip postures. This study can also cope with the online change in grasping state and flexible tool, hand, and body structures.
• Figure 2: The overall software system: the network structure of TBNPB, network trainer of TBNPB, online grasping state updater through parametric bias, tool-tip state estimator, and tool-tip controller.
• Figure 3: The tools used in this study: normal and connected dusters.
• Figure 4: The robots used in this study: PR2 with the parallel gripper and the musculoskeletal arm MusashiLarm with the flexible hand.
• Figure 5: Parametric bias trained in PR2 simulation experiment.