Enhancing Dexterity in Confined Spaces: Real-Time Motion Planning for Multi-Fingered In-Hand Manipulation

TL;DR

This work tackles real-time, high-dimensional in-hand manipulation with multi-fingered hands by learning a neural-network-based C-space that estimates collision distances and gradients for self-collision and hand-object contact. It combines DS-based obstacle avoidance with sampling-based planners (DS-guided RRT* and Dynamic PRM*) to generate collision-free, dynamic trajectories, including in-hand sliding along object surfaces. Key contributions include a generalized implicit C-space representation, a fast reactive planning framework, and online dynamic path optimization validated on a 16-DoF Allegro hand in both simulation and real-robot experiments, even with deformable objects. The approach enables faster, safer dexterous manipulation in cluttered and dynamic environments, with potential applications in advanced grasp reconfigurations and sliding-based manipulation tasks.

Abstract

Dexterous in-hand manipulation in robotics, particularly with multi-fingered robotic hands, poses significant challenges due to the intricate avoidance of collisions among fingers and the object being manipulated. Collision-free paths for all fingers must be generated in real-time, as the rapid changes in hand and finger positions necessitate instantaneous recalculations to prevent collisions and ensure undisturbed movement. This study introduces a real-time approach to motion planning in high-dimensional spaces. We first explicitly model the collision-free space using neural networks that are retrievable in real time. Then, we combined the C-space representation with closed-loop control via dynamical system and sampling-based planning approaches. This integration enhances the efficiency and feasibility of path-finding, enabling dynamic obstacle avoidance, thereby advancing the capabilities of multi-fingered robotic hands for in-hand manipulation tasks.

Figures (11)

• Figure 1: Examples of challenging in-hand manipulation motions demonstrated on a 16-DoF Allegro robotic hand consisting of four fingers (4-DoF each) and a palm. Failures often happen due to (a) finger-object collision or (b) finger-finger collision. Our proposed algorithm enables (c) successful regrasp and (d) finger sliding motion, by generating collision-free motions of multiple fingers in real time.
• Figure 2: Framework for applying our proposed real-time motion planning algorithms.
• Figure 3: Example of DS-based motion planning. The left figure shows the side view of the start (solid, finger straight) and the target (transparent, finger most bend) hand configurations. The middle finger must bend (dashed arrow line) to avoid a spherical obstacle indicated by a red dot. The right figure visualizes the isolines and DS streamlines in joint C-space in a 2D space ($q_2$ and $q_3$ of the finger). The black curve is the obstacle boundary. The red triangle and yellow star indicate the start and end configurations in the joint C-space. The color bar refers to the normalized spatial distance between the obstacle and the finger at the corresponding configuration. DS streamlines illustrate the velocity field.
• Figure 4: Evaluating the Accuracy of Neural Networks with Different Architectures through Cross-Validation. The bar graph displays the mean accuracy of 20 trials of various neural network structures, each trained for 10,000 epochs. The height of each bar indicates the mean accuracy, while the error bars represent the standard deviation. The X-axis labels denote each neural network structure's size (number of neurons). The performance no longer significantly improves for NN topologies larger than [350, 175, 90], which is hence selected in designing our neural network.
• Figure 5: Heatmap for a grid search on the false positive rate (FP). The goal is to determine the optimal values of the weights $w_2$ and $w_3$ in the cost function given by Eq. \ref{['eq:NN_1:cost']}. The $(w_2, w_3)$ pair with the lowest FP is highlighted in white $(w_2=6, w_3=4)$, and used in our self-collision model.