Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Supervised Learning of Grasping Arbitrary Objects On-the-Move

Takuya Kiyokawa, Eiki Nagata, Yoshihisa Tsurumine, Yuhwan Kwon, Takamitsu Matsubara

TL;DR

This study introduces three fully convolutional neural network models to predict static grasp primitive, dynamic grasp primitive, and residual moving velocity error from visual inputs and demonstrates that the ablation study demonstrated that the proposed method achieved the highest grasping accuracy and pick-and-place efficiency.

Abstract

Mobile grasping enhances manipulation efficiency by utilizing robots' mobility. This study aims to enable a commercial off-the-shelf robot for mobile grasping, requiring precise timing and pose adjustments. Self-supervised learning can develop a generalizable policy to adjust the robot's velocity and determine grasp position and orientation based on the target object's shape and pose. Due to mobile grasping's complexity, action primitivization and step-by-step learning are crucial to avoid data sparsity in learning from trial and error. This study simplifies mobile grasping into two grasp action primitives and a moving action primitive, which can be operated with limited degrees of freedom for the manipulator. This study introduces three fully convolutional neural network (FCN) models to predict static grasp primitive, dynamic grasp primitive, and residual moving velocity error from visual inputs. A two-stage grasp learning approach facilitates seamless FCN model learning. The ablation study demonstrated that the proposed method achieved the highest grasping accuracy and pick-and-place efficiency. Furthermore, randomizing object shapes and environments in the simulation effectively achieved generalizable mobile grasping.

Self-Supervised Learning of Grasping Arbitrary Objects On-the-Move

TL;DR

This study introduces three fully convolutional neural network models to predict static grasp primitive, dynamic grasp primitive, and residual moving velocity error from visual inputs and demonstrates that the ablation study demonstrated that the proposed method achieved the highest grasping accuracy and pick-and-place efficiency.

Abstract

Mobile grasping enhances manipulation efficiency by utilizing robots' mobility. This study aims to enable a commercial off-the-shelf robot for mobile grasping, requiring precise timing and pose adjustments. Self-supervised learning can develop a generalizable policy to adjust the robot's velocity and determine grasp position and orientation based on the target object's shape and pose. Due to mobile grasping's complexity, action primitivization and step-by-step learning are crucial to avoid data sparsity in learning from trial and error. This study simplifies mobile grasping into two grasp action primitives and a moving action primitive, which can be operated with limited degrees of freedom for the manipulator. This study introduces three fully convolutional neural network (FCN) models to predict static grasp primitive, dynamic grasp primitive, and residual moving velocity error from visual inputs. A two-stage grasp learning approach facilitates seamless FCN model learning. The ablation study demonstrated that the proposed method achieved the highest grasping accuracy and pick-and-place efficiency. Furthermore, randomizing object shapes and environments in the simulation effectively achieved generalizable mobile grasping.

Paper Structure

This paper contains 17 sections, 2 equations, 17 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Robotic Tidy-Up Tasks
  4. Self-Supervised Grasp Learning
  5. Dynamic Grasping
  6. Proposed Method
  7. Overview
  8. Motion Primitives Defined for a Mobile Grasping Action
  9. Self-Supervised Mobile Grasping Learning
  10. Training and Evaluating Models
  11. Comparative Methods
  12. Grasping Success Rate Comparisons
  13. Learning Stable Mobile Grasping
  14. Evaluating Real-World Mobile Grasping
  15. Grasping Success Rate Comparisons
  16. ...and 2 more sections

Figures (17)

  • Figure 1: Training in simulations
  • Figure 2: Execution in the real world
  • Figure 4: Overview of the mobile grasping action generator. The models of the red boxes are first trained for the stationary object grasping action policy. These are then frozen, and the other models of the blue boxes are trained to learn the moving and dynamic grasping action policies.
  • Figure 5: Coordinate systems defined in the experimental environment
  • Figure 6: Simulation
  • ...and 12 more figures