Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Hand Gesture Recognition for Robot Manipulator Tasks

Dharmendra Sharma, Peeyush Thakur, Sandeep Gupta, Narendra Kumar Dhar, Laxmidhar Behera

TL;DR

This paper proposes a novel approach to recognizing dynamic hand gestures facilitating seamless interaction between humans and robots using the proposed unsupervised model based on the Gaussian Mixture model.

Abstract

This paper proposes a novel approach to recognizing dynamic hand gestures facilitating seamless interaction between humans and robots. Here, each robot manipulator task is assigned a specific gesture. There may be several such tasks, hence, several gestures. These gestures may be prone to several dynamic variations. All such variations for different gestures shown to the robot are accurately recognized in real-time using the proposed unsupervised model based on the Gaussian Mixture model. The accuracy during training and real-time testing prove the efficacy of this methodology.

Dynamic Hand Gesture Recognition for Robot Manipulator Tasks

TL;DR

This paper proposes a novel approach to recognizing dynamic hand gestures facilitating seamless interaction between humans and robots using the proposed unsupervised model based on the Gaussian Mixture model.

Abstract

This paper proposes a novel approach to recognizing dynamic hand gestures facilitating seamless interaction between humans and robots. Here, each robot manipulator task is assigned a specific gesture. There may be several such tasks, hence, several gestures. These gestures may be prone to several dynamic variations. All such variations for different gestures shown to the robot are accurately recognized in real-time using the proposed unsupervised model based on the Gaussian Mixture model. The accuracy during training and real-time testing prove the efficacy of this methodology.
Paper Structure (16 sections, 6 equations, 4 figures, 4 tables, 2 algorithms)

This paper contains 16 sections, 6 equations, 4 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Literature Review and Problem formulation
  3. Problem formulation
  4. Proposed Methodology
  5. System Setup
  6. Hand Gesture Recording and Initial Processing
  7. Gesture Identification
  8. Features from each Video
  9. GMM Training on Features
  10. Gesture Assignment based on Features
  11. Evaluation Metrics
  12. Convergence of Proposed Methodology
  13. Results and Discussions
  14. Experimental Setup
  15. Experiments
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Real-time hand gesture recognition using an RGB-D camera attached to the robot manipulator.
  • Figure 3: Four dynamic hand gestures: (a) wave: movement largely in $x-y$ directions, (c) pick: movement largely in $y$ direction, (e) stack: movement in all $x-y-z$ directions, and (g) push: movement largely in $z$ direction. The robot tasks are: (b) initializing the robot gripper, (d) picking up an object, (f) stacking an object on a box, and (h) pushing the object. A video demonstrating all these tasks can be seen at youtube.
  • Figure 4: Model training on gesture data for four tasks.(a) Before training: the gesture data were not segregated in accurate clusters. The features (variance in x and y) can not distinguish different gesture types. (b) After training: the features now aligned each gesture data in appropriate clusters.
  • Figure 5: Model testing on real-time gestures of four tasks. (a) variance largely in $y$ than $x$ direction, (b) variance distributed in both $x-y$ directions, (c) variance follows a uniform pattern in $x-y$ directions, and (d) variance largely in $y$ than $x$ direction.