GAMMA: Generalizable Articulation Modeling and Manipulation for Articulated Objects

TL;DR

This paper proposes a novel framework of Generalizable Articulation Modeling and Manipulating for Articulated Objects (GAMMA), which learns both articulation modeling and grasp pose affordance from diverse articulated objects with different categories.

Abstract

Articulated objects like cabinets and doors are widespread in daily life. However, directly manipulating 3D articulated objects is challenging because they have diverse geometrical shapes, semantic categories, and kinetic constraints. Prior works mostly focused on recognizing and manipulating articulated objects with specific joint types. They can either estimate the joint parameters or distinguish suitable grasp poses to facilitate trajectory planning. Although these approaches have succeeded in certain types of articulated objects, they lack generalizability to unseen objects, which significantly impedes their application in broader scenarios. In this paper, we propose a novel framework of Generalizable Articulation Modeling and Manipulating for Articulated Objects (GAMMA), which learns both articulation modeling and grasp pose affordance from diverse articulated objects with different categories. In addition, GAMMA adopts adaptive manipulation to iteratively reduce the modeling errors and enhance manipulation performance. We train GAMMA with the PartNet-Mobility dataset and evaluate with comprehensive experiments in SAPIEN simulation and real-world Franka robot. Results show that GAMMA significantly outperforms SOTA articulation modeling and manipulation algorithms in unseen and cross-category articulated objects. We will open-source all codes and datasets in both simulation and real robots for reproduction in the final version. Images and videos are published on the project website at: http://sites.google.com/view/gamma-articulation

Figures (5)

• Figure 1: GAMMA framework in real microwave task. (a) Real microwave image to generate point clouds. (b) Articulated structure modeling, where blue points are segmented revolute joint and red arrow is the estimated but inaccurate joint axis and origin. (c) Grasp pose affordance evaluates the actionability and chooses ideal grasp poses. (d) Adaptive manipulation to iteratively update joint parameters. The initial joint axis(red) constantly gets closer to the ground truth after several iterations.
• Figure 2: Illustration of points and vectors on articulated objects.
• Figure 3: Pipeline of GAMMA. We collect RGB-D images of articulated objects like a cabinet to generate point clouds. The articulation modeling block segments the articulated parts and estimates the joint parameters. The grasp pose affordance block estimates the actionability of each grasp pose and chooses the ideal ones. In the adaptive manipulation, the articulation model provides open-loop trajectory planning and we iteratively update the joint parameters with actual trajectory to improve modeling accuracy and grasping success rate.
• Figure 4: We implement ANCSH and GAMMA to model articulated objects in 7 categories. The first rows are images in the simulation environment. The second and third rows are segmentation results of articulated parts, marked as blue, green and dark green points. Each color represents a separate modeled articulated part. The red arrow and dot denote the estimated joint axis direction and origin position.
• Figure 5: We implement GAMMA in the real-world experiments. Manipulation tasks include pulling the drawer and door on a cabinet and pulling the door on a microwave.