P3-SAM: Native 3D Part Segmentation

TL;DR

The paper addresses the challenge of robust, fully automatic 3D part segmentation for arbitrary objects. It introduces P3-SAM, a native 3D point-promptable model with a feature extractor, three segmentation heads, and an IoU predictor, trained on a large-scale 3D dataset and equipped with an automatic mask-merging pipeline. Empirical results show state-of-the-art performance across multiple datasets and tasks, including non-watertight meshes, with robust automatic segmentation and interactive capabilities. The work shifts away from 2D data lifting, enabling scalable, versatile 3D part segmentation applicable to interactive, hierarchical, and multi-prompt workflows.

Abstract

Segmenting 3D assets into their constituent parts is crucial for enhancing 3D understanding, facilitating model reuse, and supporting various applications such as part generation. However, current methods face limitations such as poor robustness when dealing with complex objects and cannot fully automate the process. In this paper, we propose a native 3D point-promptable part segmentation model termed P$^3$-SAM, designed to fully automate the segmentation of any 3D objects into components. Inspired by SAM, P$^3$-SAM consists of a feature extractor, multiple segmentation heads, and an IoU predictor, enabling interactive segmentation for users. We also propose an algorithm to automatically select and merge masks predicted by our model for part instance segmentation. Our model is trained on a newly built dataset containing nearly 3.7 million models with reasonable segmentation labels. Comparisons show that our method achieves precise segmentation results and strong robustness on any complex objects, attaining state-of-the-art performance. Our project page is available at https://murcherful.github.io/P3-SAM/.

Figures (14)

• Figure 1: P$^3$-SAM produces precise part segmentation results for any object.
• Figure 2: The Network Architecture of P$^3$-SAM. Input point clouds are fed to feature extractor to obtain point-wise features. The features, point prompts, and original point clouds are then fed to a two stage multi-mask segmentor to obtain three masks in various scales. Finally, the IoU predictor is utilized to evaluate the quality of the masks and select the best one as the final prediction.
• Figure 3: Automatic Segmentation Pipeline. Point prompts are sampled by FPS and go through the P$^3$-SAM to obtain multiple masks. NMS is then adopted to merge redundant masks. The point-level masks are then projected onto mesh faces to obtain the part segmentation results.
• Figure 4: The comparison of our method across different tasks.
• Figure 5: The three applications of our method.