ModelNet-O: A Large-Scale Synthetic Dataset for Occlusion-Aware Point Cloud Classification

TL;DR

This work addresses the gap between synthetic clean-point-cloud benchmarks and real-world occlusion by introducing ModelNet-O, a large-scale occluded dataset generated from ModelNet40 via fixed-camera projections. It then presents PointMLS, a robust occlusion-aware classifier built on a differentiable Critical Point Sampling (CPS) module, a Neighborhood Feature Aggregation (FA) module, and a Multi-Level Sampling (MLS) architecture, optimized with a joint classification and Chamfer-based sampling loss. PointMLS achieves state-of-the-art results on ModelNet-O (OA = 78.9%) and remains competitive on complete datasets like ModelNet40 (≈94.0% OA) and ScanObjectNN (≈86.6% OA), while demonstrating strong robustness to noise. The combination of CPS and MLS provides a practical approach for occlusion-robust 3D point cloud classification with real-world applicability and a new benchmark for under-occlusion scenarios.

Abstract

Recently, 3D point cloud classification has made significant progress with the help of many datasets. However, these datasets do not reflect the incomplete nature of real-world point clouds caused by occlusion, which limits the practical application of current methods. To bridge this gap, we propose ModelNet-O, a large-scale synthetic dataset of 123,041 samples that emulate real-world point clouds with self-occlusion caused by scanning from monocular cameras. ModelNet-O is 10 times larger than existing datasets and offers more challenging cases to evaluate the robustness of existing methods. Our observation on ModelNet-O reveals that well-designed sparse structures can preserve structural information of point clouds under occlusion, motivating us to propose a robust point cloud processing method that leverages a critical point sampling (CPS) strategy in a multi-level manner. We term our method PointMLS. Through extensive experiments, we demonstrate that our PointMLS achieves state-of-the-art results on ModelNet-O and competitive results on regular datasets, and it is robust and effective. More experiments also demonstrate the robustness and effectiveness of PointMLS.

Figures (12)

• Figure 1: a) Comparison between occluded point clouds (ModelNet-O) and completed point clouds wu2015ModelNet40, our ModelNet-O simulates the collection of point clouds using a fixed camera. b) In the case of point sampling, previous methods hu2020RandLa-Netqi2017pointnet suffer from instability in dealing with outlier points, while the proposed CPS module is more robust to noise and can weaken the effect of occlusion. c) Our proposed PointMLS performs well on both occluded (ModelNet-O) and general (ModelNet40) point clouds.
• Figure 2: a) The overall scheme of generating occluded point clouds. Note that some objects cannot be projected to obtain a depth map under certain viewpoints. b) 20-view dodecahedral configuration. The virtual cameras are placed on the vertices of a dodecahedron encompassing the object. Blue points: training set. Orange points: testing set.
• Figure 3: Illustration of the critical point sampling (CPS) module and the feature aggregation (FA) module. The CPS module extracts features of the input point cloud $P\in \mathbb{R}^{N\times 3}$, and then obtains point-wise weight $W\in \mathbb{R}^{M\times N}$. Sampled point clouds $\tilde{P}\in \mathbb{R}^{M\times 3}$ are generated by matrix multiplication between $W$ and $P$. The FA module with residual MLPs further aggregates features of each local region belonging to the sampled point cloud. A classifier obtains the final score.
• Figure 4: Illustration of multi-level sampling (MLS).
• Figure 5: Visualization of different ratios of noise points.