Arti-PG: A Toolbox for Procedurally Synthesizing Large-Scale and Diverse Articulated Objects with Rich Annotations

TL;DR

Arti-PG tackles data scarcity in 3D articulated object understanding by introducing a procedural toolbox that synthesizes large-scale, richly annotated articulated objects. Objects are represented by a macro spatial-structure program tied to micro geometric details through point-wise correspondences, and new instances are generated by manipulating the structure and recovering details, with annotations aligned analytically. The approach yields a 26-category, 3096-object pipeline with automatic labeling, demonstrated to improve performance across vision (segmentation, pose estimation, completion) and manipulation tasks, outperforming baselines and conventional augmentation. This toolbox enables scalable data generation and broad applicability for articulated-object learning in both vision and robotics contexts, reducing data-collection costs and facilitating robust model training.

Abstract

The acquisition of substantial volumes of 3D articulated object data is expensive and time-consuming, and consequently the scarcity of 3D articulated object data becomes an obstacle for deep learning methods to achieve remarkable performance in various articulated object understanding tasks. Meanwhile, pairing these object data with detailed annotations to enable training for various tasks is also difficult and labor-intensive to achieve. In order to expeditiously gather a significant number of 3D articulated objects with comprehensive and detailed annotations for training, we propose Articulated Object Procedural Generation toolbox, a.k.a. Arti-PG toolbox. Arti-PG toolbox consists of i) descriptions of articulated objects by means of a generalized structure program along with their analytic correspondence to the objects' point cloud, ii) procedural rules about manipulations on the structure program to synthesize large-scale and diverse new articulated objects, and iii) mathematical descriptions of knowledge (e.g. affordance, semantics, etc.) to provide annotations to the synthesized object. Arti-PG has two appealing properties for providing training data for articulated object understanding tasks: i) objects are created with unlimited variations in shape through program-oriented structure manipulation, ii) Arti-PG is widely applicable to diverse tasks by easily providing comprehensive and detailed annotations. Arti-PG now supports the procedural generation of 26 categories of articulate objects and provides annotations across a wide range of both vision and manipulation tasks, and we provide exhaustive experiments which fully demonstrate its advantages. We will make Arti-PG toolbox publicly available for the community to use.

Figures (3)

• Figure 1: a. The point cloud of a washing machine. A small area of its door surface is zoomed in for a clear view of geometric details. b. Describing the object with spatial structure (bottom) and geometric details (top). The brown arrows concretely represent point-wise correspondence between points of the structure and the real point clouds. c. Naive program description of the structure in (b). The correspondence between the program and structure is indicated by the same color. Elementary primitive templates are in black font (e.g. Cylinder) and instances of elementary primitives are in colored font (e.g. door_inner). d. Program description of the structure in (b) via advanced primitive template. Advanced primitive templates are in black font (e.g. Body) and instances of advanced primitives are in colored font (e.g. body).
• Figure 2: Fig. I illustrates examples of structure manipulation. I-(a): The original structure. I-(b1-b3): Structures after being manipulated by CPA, DPA, APA respectively. I-(c): Structure after being manipulated by the combination of three alterations. Fig. II shows examples of mapping between points in CPA (a), DPA (b) and correspondence between elementary primitives in APA (c). In II-(a) and II-(b), points are analytically bounded to the primitive with parameterized coordinate representation. II-(c) depicts correspondence between elementary primitives by the same colors, such as silver bracket in both globes.
• Figure 3: Illustrations of analytically assigning labels on spatial structures of various categories with functions (described in mathematical formulas, the coordinate center is indicated by the arrow, zoom in for a clear view). We take affordable areas to grasp the object as examples of labels. a. edge of microwave door. b. lower half of handle (we can still represent such area with the same parameters and functions when the handle is rotated). c. beam of handle and top rim of cap knob. d. top rim of cap knob and center of kettle ring handle.