Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bring Your Own Grasp Generator: Leveraging Robot Grasp Generation for Prosthetic Grasping

Giuseppe Stracquadanio, Federico Vasile, Elisa Maiettini, Nicolò Boccardo, Lorenzo Natale

TL;DR

This paper presents a novel eye-in-hand prosthetic grasping system that initiates the approach-to-grasp action based on user's command and automatically configures the DoFs of a prosthetic hand and compares it with a multi-DoF prosthetic control baseline and finds that the method enables faster grasps, while simplifying the user experience.

Abstract

One of the most important research challenges in upper-limb prosthetics is enhancing the user-prosthesis communication to closely resemble the experience of a natural limb. As prosthetic devices become more complex, users often struggle to control the additional degrees of freedom. In this context, leveraging shared-autonomy principles can significantly improve the usability of these systems. In this paper, we present a novel eye-in-hand prosthetic grasping system that follows these principles. Our system initiates the approach-to-grasp action based on user's command and automatically configures the DoFs of a prosthetic hand. First, it reconstructs the 3D geometry of the target object without the need of a depth camera. Then, it tracks the hand motion during the approach-to-grasp action and finally selects a candidate grasp configuration according to user's intentions. We deploy our system on the Hannes prosthetic hand and test it on able-bodied subjects and amputees to validate its effectiveness. We compare it with a multi-DoF prosthetic control baseline and find that our method enables faster grasps, while simplifying the user experience. Code and demo videos are available online at https://hsp-iit.github.io/byogg/.

Bring Your Own Grasp Generator: Leveraging Robot Grasp Generation for Prosthetic Grasping

TL;DR

This paper presents a novel eye-in-hand prosthetic grasping system that initiates the approach-to-grasp action based on user's command and automatically configures the DoFs of a prosthetic hand and compares it with a multi-DoF prosthetic control baseline and finds that the method enables faster grasps, while simplifying the user experience.

Abstract

One of the most important research challenges in upper-limb prosthetics is enhancing the user-prosthesis communication to closely resemble the experience of a natural limb. As prosthetic devices become more complex, users often struggle to control the additional degrees of freedom. In this context, leveraging shared-autonomy principles can significantly improve the usability of these systems. In this paper, we present a novel eye-in-hand prosthetic grasping system that follows these principles. Our system initiates the approach-to-grasp action based on user's command and automatically configures the DoFs of a prosthetic hand. First, it reconstructs the 3D geometry of the target object without the need of a depth camera. Then, it tracks the hand motion during the approach-to-grasp action and finally selects a candidate grasp configuration according to user's intentions. We deploy our system on the Hannes prosthetic hand and test it on able-bodied subjects and amputees to validate its effectiveness. We compare it with a multi-DoF prosthetic control baseline and find that our method enables faster grasps, while simplifying the user experience. Code and demo videos are available online at https://hsp-iit.github.io/byogg/.

Paper Structure

This paper contains 23 sections, 5 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Monocular Depth Estimation
  4. Vision-based Prosthetic Grasping
  5. Vision-based Robotic Grasping
  6. Visual Odometry
  7. METHODS
  8. Overview
  9. Architecture Details
  10. Monocular Depth Estimation
  11. Grasp Generation
  12. Visual Odometry
  13. Mapping grasp candidates to the Hannes hand
  14. EXPERIMENTAL SETUP
  15. Deployment and Embodiment
  16. ...and 8 more sections

Figures (5)

  • Figure 1: The phases of our grasping pipeline. A demonstration of semi-autonomous grasp is shown through the user perspective.
  • Figure 2: Details of our grasping pipeline. (1) A depth $D_0$ is estimated from the first frame of the grasping sequence. (2) $D_0$ is used to build the point cloud $PCD_0$ and generate a distribution of grasp poses. (3) The hand-trajectory is estimated by a visual odometry module, and used to select a grasp candidate. (4) The candidate pose is mapped to Hannes.
  • Figure 3: AGT and GSR measured on able-bodied subjects. Statistics are shown per object (a) and per subject (b).
  • Figure 4: AGT and GSR measured on amputees. Results with our method are compared with CC-SSC and B-SSC baselines.
  • Figure 5: Distributions of pupil diameter (PD) recorded while performing trials.