ProtoSeg: A Prototype-Based Point Cloud Instance Segmentation Method

TL;DR

This paper presents a novel neural network architecture for performing instance segmentation on 3D point clouds that is not only 28% faster than the state-of-the-art, it also exhibits the lowest standard deviation.

Abstract

3D instance segmentation is crucial for obtaining an understanding of a point cloud scene. This paper presents a novel neural network architecture for performing instance segmentation on 3D point clouds. We propose to jointly learn coefficients and prototypes in parallel which can be combined to obtain the instance predictions. The coefficients are computed using an overcomplete set of sampled points with a novel multi-scale module, dubbed dilated point inception. As the set of obtained instance mask predictions is overcomplete, we employ a non-maximum suppression algorithm to retrieve the final predictions. This approach allows to omit the time-expensive clustering step and leads to a more stable inference time. The proposed method is not only 28% faster than the state-of-the-art, it also exhibits the lowest standard deviation. Our experiments have shown that the standard deviation of the inference time is only 1.0% of the total time while it ranges between 10.8 and 53.1% for the state-of-the-art methods. Lastly, our method outperforms the state-of-the-art both on S3DIS-blocks (4.9% in mRec on Fold-5) and PartNet (2.0% on average in mAP).

Figures (8)

• Figure 1: Speed-performance comparison on S3DIS-blocks Area-5. The proposed method outperforms the state-of-the-art in terms of accuracy, speed and variance in inference time.
• Figure 2: The ProtoSeg network architecture consists of four main parts: (1) A feature extractor which retrieves per-point features. (2) A point sampler, obtaining points with large diversity in feature space. (3) ProtoScoreNet, which retrieves a set of prototypes. In parallel, (4) Coeffnet computes for each sampled point a set of coefficients associated with the prototypes. To allow multi-scale coefficients retrieval, a Dilated Point Inception (DPI) module is employed. Instance predictions are obtained by linearly combining the coefficients and prototypes.
• Figure 3: The different scales of DPI. From left to right: the sampled points and the receptive fields for the DPI branch with dilation factor 1 and 8, respectively.
• Figure 4: Distribution of the computed coefficients by ProtoSeg on the test set of PartNet level-1. Only the coefficients leading to an instance which is retained by NMS, are taken into account.
• Figure 5: Prototype $p_i$ activation for different PartNet samples $s_i$.