Instance Segmentation for Point Sets

TL;DR

This work tackles the memory bottleneck of SGPN, which requires an $N \times N$ similarity matrix for instance segmentation on 3D point sets. It introduces two sub-sampling strategies—Random Sub-sampling and Grid-based Sub-sampling—to operate on a fixed landmark subset of size $K$, then propagate labels to the full point set via nearest-neighbor rules, effectively reducing memory from $O(N^2)$ to $O(K^2)$ and speeding up computation. Experiments on the Stanford Indoor 3D dataset show that sub-sampling can achieve near full-set performance with substantially lower memory and time costs, with random sub-sampling performing best at smaller $K$ while grid-based sampling can better capture certain patterns at comparable $K$. These techniques enable scalable 3D point-set instance segmentation, offering practical routes to deploy PointNet++-based methods on larger scenes. The approaches also open avenues for future work, such as propagation in feature space and alternative landmark selection strategies.

Abstract

Recently proposed neural network architectures like PointNet [QSMG16] and PointNet++ [QYSG17] have made it possible to apply Deep Learning to 3D point sets. The feature representations of shapes learned by these two networks enabled training classifiers for Semantic Segmentation, and more recently for Instance Segmentation via the Similarity Group Proposal Network (SGPN) [WYHN17]. One area of improvement which has been highlighted by SGPN's authors, pertains to use of memory intensive similarity matrices which occupy memory quadratic in the number of points. In this report, we attempt to tackle this issue through use of two sampling based methods, which compute Instance Segmentation on a sub-sampled Point Set, and then extrapolate labels to the complete set using the nearest neigbhour approach. While both approaches perform equally well on large sub-samples, the random-based strategy gives the most improvements in terms of speed and memory usage.

Figures (10)

• Figure 1: Overlapping Blocks shown for one of the scenes (This is top view of the scene).
• Figure 2: Random sub-sampling method on the left misses an entire object on the left as it focuses on high density parts of the scene whereas these points would be captured by the grid-based sub-sampling method as shown on the right.
• Figure 3: Two density configurations of the grid with the corresponding chosen landmarks for the same scene. On the left, the n-neighbours create small clustering patterns due to the small density of the grid in comparison with the right configuration.
• Figure 4: Finding $K=1024$ landmarks for the same scene block using algorithm \ref{['grid-algo']} on the left and algorithm \ref{['gridext-algo']} on the right. A grid size of $1024$ was sufficient to produce the results on the left whereas $4096$ points were needed for the same-quality result on the right.
• Figure 5: Random Sub-sampling: Mean Average Precision for Instance Segmentation for three scenes from the Stanford Indoor 3D Dataset stanford-dataset.