G-FARS: Gradient-Field-based Auto-Regressive Sampling for 3D Part Grouping

TL;DR

This work introduces 3D part grouping, a task that requires identifying all valid groupings of mixed parts from multiple shapes. It proposes G-FARS, a gradient-field-based auto-regressive sampling framework that uses a gradient-field-based selection GNN to model the conditional distribution over part selections and to autoregressively extract groups from a mixed set. Grounded in score-based modeling with stochastic differential equations, the method trains to approximate the gradient of the conditional log-density and employs a Predictor-Corrector sampler to generate groups. Experiments on PartNet-derived datasets show that G-FARS outperforms GRU-Mask and Comp-Net baselines, and ablations confirm the importance of the GF and GNN components as well as the PC sampling strategy. The approach enables zero-shot noisy-part removal and demonstrates promising potential for automated sorting and assembly in industrial and recycling contexts, with planned future work toward real-world robotic deployment and broader generalization.

Abstract

This paper proposes a novel task named "3D part grouping". Suppose there is a mixed set containing scattered parts from various shapes. This task requires algorithms to find out every possible combination among all the parts. To address this challenge, we propose the so called Gradient Field-based Auto-Regressive Sampling framework (G-FARS) tailored specifically for the 3D part grouping task. In our framework, we design a gradient-field-based selection graph neural network (GNN) to learn the gradients of a log conditional probability density in terms of part selection, where the condition is the given mixed part set. This innovative approach, implemented through the gradient-field-based selection GNN, effectively captures complex relationships among all the parts in the input. Upon completion of the training process, our framework becomes capable of autonomously grouping 3D parts by iteratively selecting them from the mixed part set, leveraging the knowledge acquired by the trained gradient-field-based selection GNN. Our code is available at: https://github.com/J-F-Cheng/G-FARS-3DPartGrouping.

Figures (9)

• Figure 1: The definition of our proposed 3D part grouping task. Assuming we have a set which contains mixed parts from $N$ shapes. Our goal in this task is to use a grouping algorithm to separate these mixed parts and group them by their respective shapes.
• Figure 2: The auto-regressive sampling procedure of our proposed framework. First, we obtain the per-part feature by using a PointNet qi2017pointnet to encode all the input parts $P_n$ at the iteration $n$. Then, we use the gradient-field-based (G-F-based) selection GNN to sample a selection vector $\mathbf{c}_n$ to obtain part group $n$, and use $\overline{\mathbf{c}_n}$ to get the rest parts $P_{n+1}$. $P_{n+1}$ will be the next input parts at the next iteration. The auto-regressive sampling stops when $P_{n+1}$ is empty, or $n$ reaches the maximum iteration.
• Figure 3: The random mixing method to create a mixed part set. We randomly selected $N$ shapes from the PartNet dataset. We then mix all the parts into a single set. The sequence of parts are shuffled.
• Figure 4: The qualitative comparisons. To intuitively demonstrate the effect of grouping, we rotate and translate all the parts by using their ground truth poses after the grouping procedure. The results show that only G-FARS is able to correctly group the 3D parts. Some baselines even predict an incorrect number of groups (e.g., Comp-Net). Due to the page limit, we here only present the results of G-FARS and the two most competitive baselines (i.e., GRU-Mask and Comp-Net). We have included the full comparison and an additional qualitative comparison figure in the supplementary materials.
• Figure 5: Three examples of noisy part removal.