Attention-based Estimation and Prediction of Human Intent to augment Haptic Glove aided Control of Robotic Hand

TL;DR

A control algorithm is presented to transform the synthesized intent at the RH and allow relocation of the object to the expected goal pose and an attention-based convolutional neural network encoder is leverage to predict the trajectory of intent for a certain lookahead to compensate for the delays.

Abstract

The letter focuses on Haptic Glove (HG) based control of a Robotic Hand (RH) executing in-hand manipulation of certain objects of interest. The high dimensional motion signals in HG and RH possess intrinsic variability of kinematics resulting in difficulty to establish a direct mapping of the motion signals from HG onto the RH. An estimation mechanism is proposed to quantify the motion signal acquired from the human controller in relation to the intended goal pose of the object being held by the robotic hand. A control algorithm is presented to transform the synthesized intent at the RH and allow relocation of the object to the expected goal pose. The lag in synthesis of the intent in the presence of communication delay leads to a requirement of predicting the estimated intent. We leverage an attention-based convolutional neural network encoder to predict the trajectory of intent for a certain lookahead to compensate for the delays. The proposed methodology is evaluated across objects of different shapes, mass, and materials. We present a comparative performance of the estimation and prediction mechanisms on 5G-driven real-world robotic setup against benchmark methodologies. The test-MSE in prediction of human intent is reported to yield ~ 97.3 -98.7% improvement of accuracy in comparison to LSTM-based benchmark

Figures (7)

• Figure 1: The proposed system workflow for estimation & the prediction of human intent with corresponding control mechanism. The estimation mechanism (left) is detailed in Sections \ref{['sec:estimationintent']} and \ref{['sec:predictedintent']}; The control algorithm (right) is detailed in Section \ref{['sec:controlsection']}.
• Figure 2: Illustration of a tree-type robotic hand holding an object of interest to undergo in-hand manipulation from the current/achieved pose to the desired pose.
• Figure 3: Attention-based proposed architecture for Intent prediction.
• Figure 4: PCA analysis of the data corresponding for exemplar manipulation of different objects.
• Figure 5: Motion achieved on ARH as a result of estimation, and control mechanisms while manipulating various objects under study about the three Cartesian axes. First two rows show snapshots of pose achieved during rotation about $x$-axis, the next two rows show pose achieved while rotating about $y$-axis and the last two rows show pose achieved while rotating about $z$-axis achieved as a result of proposed methodology while moving from initial pose () towards desired goal pose through intermediate pose (). The results show rotations about the three Cartesian axes using multiple types of objects viz., (Hollow metal cube), (Hollow metal cylinder), (Hollow metal sphere), (Solid plastic cube), (Hollow plastic cylinder), (Solid plastic sphere), (Solid wooden cube), (Solid wooden cylinder), (Solid wooden sphere), (Hollow cardboard cube), (Rubber ball), and (Cylindrical transparent plastic bottle), illustrating the robustness of the proposed approach across various shapes and materials of objects.