Kinesthetic-based In-Hand Object Recognition with an Underactuated Robotic Hand

TL;DR

This work shows that an inexpensive tendon-based underactuated hand can recognize in-hand objects using only kinesthetic signals from actuators, without tactile sensors or vision. By collecting sequential proprioceptive data during manipulation and training temporal models such as LSTMs or TCNs, the authors achieve high recognition accuracy (approaching 95%) and leverage real-time majority voting to improve certainty. The approach generalizes beyond specific objects to shape categories, enabling robust recognition under occlusion or absence of visual feedback. Practically, this kinesthetic-haptic framework provides a simple, low-cost enhancement for in-hand perception that can complement or substitute vision in constrained environments.

Abstract

Tendon-based underactuated hands are intended to be simple, compliant and affordable. Often, they are 3D printed and do not include tactile sensors. Hence, performing in-hand object recognition with direct touch sensing is not feasible. Adding tactile sensors can complicate the hardware and introduce extra costs to the robotic hand. Also, the common approach of visual perception may not be available due to occlusions. In this paper, we explore whether kinesthetic haptics can provide in-direct information regarding the geometry of a grasped object during in-hand manipulation with an underactuated hand. By solely sensing actuator positions and torques over a period of time during motion, we show that a classifier can recognize an object from a set of trained ones with a high success rate of almost 95%. In addition, the implementation of a real-time majority vote during manipulation further improves recognition. Additionally, a trained classifier is also shown to be successful in distinguishing between shape categories rather than just specific objects.

Figures (11)

• Figure 1: In-hand recognition of an object in an occluded environment with an underactuated hand where visual perception is not available. The hand relies on kinesthetic perception during in-hand manipulation to either recognize the specific object from a trained set or its general shape.
• Figure 2: An underactuated robotic hand (OpenHand Model-O Ma2017YaleOP) with two-fingers. The hand is mostly 3D printed and is tendon-based. Each finger consists of two passive joints with springs. The tendons run along the length of the two fingers and are pulled by two actuators at the base of the hand.
• Figure 3: Eight distinct objects used for training the classification model.
• Figure 4: Confusion matrix for classification of the eight objects with the TCN model over the test data.
• Figure 5: Object recognition success rate with regards to the fraction of total training data used to train RF, LSTM, SVM, FC-NN and TCN.