Stable Tool-Use with Flexible Musculoskeletal Hands by Learning the Predictive Model of Sensor State Transition

TL;DR

The paper tackles stable tool-use with a flexible musculoskeletal hand by learning a predictive model of sensor-state transitions from real robot data and using it to drive a grasping stabilizer. A fixed-horizon neural predictor with $T=10$ predicts $(2)$ seconds ahead at 5 Hz, trained via random search data with the loss $L_{origin}$, while a task-specific loss $L_{grasp}$ and optimization $L_{opt}$ guide a gradient-based input update to keep the initial contact state $m{s}^{keep}$. The grasping stabilizer, built from batches of initial inputs and gradient steps, is validated on hammering, vacuuming, and brooming with Musashi, showing improved stability (lower $L_{eval}$ and sustained contact) compared with no stabilizer or constant-search data. The results demonstrate that implicit modeling of sensor-actuator dynamics enables robust tool-use without full physics models, paving the way for generalization to other tools and hands. Future work aims to extend to multiple tools and in-hand manipulation with higher-speed control and richer muscle coordination.

Abstract

The flexible under-actuated musculoskeletal hand is superior in its adaptability and impact resistance. On the other hand, since the relationship between sensors and actuators cannot be uniquely determined, almost all its controls are based on feedforward controls. When grasping and using a tool, the contact state of the hand gradually changes due to the inertia of the tool or impact of action, and the initial contact state is hardly kept. In this study, we propose a system that trains the predictive network of sensor state transition using the actual robot sensor information, and keeps the initial contact state by a feedback control using the network. We conduct experiments of hammer hitting, vacuuming, and brooming, and verify the effectiveness of this study.

Figures (11)

• Figure 1: Grasping stabilizer for tool-use.
• Figure 2: Five-fingered musculoskeletal flexible hand makino2018hand installed in the musculoskeletal humanoid Musashi kawaharazuka2019musashi.
• Figure 3: Experimental evaluation of the correlation between $L_{eval}$ and sound value when hitting a plate with a hammer.
• Figure 4: Experiments of hammer hitting, vacuuming, and brooming.
• Figure 5: Transition of $\bm{u}$ during random search behaviors: Variable and Constant Search.