RPMArt: Towards Robust Perception and Manipulation for Articulated Objects

TL;DR

A framework towards Robust Perception and Manipulation for Articulated Objects (RPMArt), which learns to estimate the articulation parameters and manipulate the articulation part from the noisy point cloud, and introduces an articulation-aware classification scheme to enhance its ability for sim-to-real transfer.

Abstract

Articulated objects are commonly found in daily life. It is essential that robots can exhibit robust perception and manipulation skills for articulated objects in real-world robotic applications. However, existing methods for articulated objects insufficiently address noise in point clouds and struggle to bridge the gap between simulation and reality, thus limiting the practical deployment in real-world scenarios. To tackle these challenges, we propose a framework towards Robust Perception and Manipulation for Articulated Objects (RPMArt), which learns to estimate the articulation parameters and manipulate the articulation part from the noisy point cloud. Our primary contribution is a Robust Articulation Network (RoArtNet) that is able to predict both joint parameters and affordable points robustly by local feature learning and point tuple voting. Moreover, we introduce an articulation-aware classification scheme to enhance its ability for sim-to-real transfer. Finally, with the estimated affordable point and articulation joint constraint, the robot can generate robust actions to manipulate articulated objects. After learning only from synthetic data, RPMArt is able to transfer zero-shot to real-world articulated objects. Experimental results confirm our approach's effectiveness, with our framework achieving state-of-the-art performance in both noise-added simulation and real-world environments. Code, data and more results can be found on the project website at https://r-pmart.github.io.

Figures (7)

• Figure 1: RPMArt framework to tackle the real-world articulated objects perception and manipulation. (a) During training, voting targets are generated by part segmentation, joint parameters and affordable points from the simulator to supervise RoArtNet. (b) Given the real-world noisy point cloud observation, RoArtNet can still generate robust joint parameters and affordable points estimation by point tuple voting. Then, affordable initial grasp poses can be selected from AnyGrasp-generated grasp poses based on the estimated affordable points, and subsequent actions can be constrained by the estimated joint parameters.
• Figure 2: Illustration of joint parameters and affordable points on articulated objects.
• Figure 3: Overview of RoArtNet. First, (a) a collection of $M$-point tuples ($M=3$ here as an example) are uniformly sampled from the point cloud. For each point tuple, (b) we predict several voting targets with a neural network from the local context features of the point tuple. Further, an articulation score $c$ is applied to supervise the neural network so that the network is aware of the articulation structure. Then, (c) we can generate multiple candidates using the predicted voting targets, given the one degree-of-freedom ambiguity constraint. (d) The candidate joint origin, joint direction and affordable point with the most votes, from only point tuples with high articulation score, are selected as the final estimation.
• Figure 4: Articulation perception results. We gradually add higher level of noise to the input point clouds, and test the joint parameters and affordable points estimation performance. Lower is better. Results are averaged across six object categories. Error bars represent the standard deviation. Different noise levels are detailed in Sec. \ref{['parag:noise-perception']}. More detailed results for each category are listed on our website.
• Figure 5: Articulated object manipulation results. We report the success rate averaged among around 100 trials per object instance for each task. Higher is better. Selected noise levels are detailed in Sec. \ref{['parag:noise-manipulation']}. More results for other tasks are shown on our website.