Systematically Exploring the Landscape of Grasp Affordances via Behavioral Manifolds

TL;DR

This work proposes a different perspective on grasp affordance learning, explicitly accounting for grasp synthesis; that is, the manner in which manipulator kinematics are used to allow materialization of grasps.

Abstract

The use of machine learning to investigate grasp affordances has received extensive attention over the past several decades. The existing literature provides a robust basis to build upon, though a number of aspects may be improved. Results commonly work in terms of grasp configuration, with little consideration for the manner in which the grasp may be (re-)produced from a reachability and trajectory planning perspective. In addition, the majority of existing learning approaches focus of producing a single viable grasp, offering little transparency on how the result was reached, or insights on its robustness. We propose a different perspective on grasp affordance learning, explicitly accounting for grasp synthesis; that is, the manner in which manipulator kinematics are used to allow materialization of grasps. The approach allows to explicitly map the grasp policy space in terms of generated grasp types and associated grasp quality. Results of numerical simulations illustrate merit of the method and highlight the manner in which it may promote a greater degree of explainability for otherwise intransparent reinforcement processes.

Figures (9)

• Figure 1: Considered end-effector (Robotiq 3-digit gripper), shown example behavioral manifolds for a hand-drill (from the YCB dataset) for the considered gripper and grasp synthesis policy discussed in Section \ref{['sec:grasp_synthesis']} (left); grasping a cylinder, actuated joints, centers of mass of the cylinder and gripper base, and contact points shown in blue (right).
• Figure 2: Horizontal cut of behavioral map for a cup (left), grasped with a Robotiq 3-fingered gripper, behavioral manifolds $\mathcal{A}$ through $\mathcal{D}$ map to consistent grasp behaviors (right). Contours inside manifolds show $\mu_{\rm s}$ level sets, local minima marked with yellow dots. The red line shows an example optimization trajectory from PPO (jumps in light green). Vertical cut shown bottom right, 3D perspective bottom far right. The behavioral map provides a holistic perspective of possible grasps.
• Figure 3: Static ($\mu_{\rm g}$) and dynamic ($\mu_{\rm s}$) grasp metrics as a function of maximum joint torques, grasping a rectangular plate. Grasps learned using the static metric in the top row (grasps a, b), dynamic metric in the bottom row (c, d).
• Figure 4: Proximity perception is used to conform shape of the gripper to that of the object, links in closer proximity move away, those further away move closer (left). Exploiting proximity information expands behavioral manifolds (right).
• Figure 5: Grasps produced by $\mathcal{G}_{\rm c}$ (left) and $\mathcal{G}_{\rm s}$ (right) from the same initial relative pose; $\mathcal{G}_{\rm s}$ results in more resilient grasps.