Unsupervised Occupancy Learning from Sparse Point Cloud

TL;DR

This work tackles unsupervised 3D shape reconstruction from sparse, noisy, unoriented point clouds by learning occupancy fields with an implicit neural representation. It introduces a margin-based uncertainty sampling loss using $U_\theta(x)=P(y=1|x,\theta)-P(y=0|x,\theta)$ and a Newton-Raphson style update to align the occupancy boundary with observed surface points, complemented by an entropy-based regularization that promotes low entropy away from the surface and high entropy near the surface. The training objective combines $\mathcal{L}_{\text{samp}}$ with $\mathcal{L}_{entr}$, decaying the entropy weight over time, and results are extracted as meshes via Marching Cubes. Extensive experiments on ShapeNet, real articulated data, and large-scale scenes show that occupancy-based, unsupervised learning outperforms SDF-based and prior occupancy methods under sparse inputs, improving both quantitative metrics and qualitative reconstructions. This approach enhances robustness and generalization for practical 3D reconstruction in settings with limited, noisy data.

Abstract

Implicit Neural Representations have gained prominence as a powerful framework for capturing complex data modalities, encompassing a wide range from 3D shapes to images and audio. Within the realm of 3D shape representation, Neural Signed Distance Functions (SDF) have demonstrated remarkable potential in faithfully encoding intricate shape geometry. However, learning SDFs from 3D point clouds in the absence of ground truth supervision remains a very challenging task. In this paper, we propose a method to infer occupancy fields instead of SDFs as they are easier to learn from sparse inputs. We leverage a margin-based uncertainty measure to differentially sample from the decision boundary of the occupancy function and supervise the sampled boundary points using the input point cloud. We further stabilize the optimization process at the early stages of the training by biasing the occupancy function towards minimal entropy fields while maximizing its entropy at the input point cloud. Through extensive experiments and evaluations, we illustrate the efficacy of our proposed method, highlighting its capacity to improve implicit shape inference with respect to baselines and the state-of-the-art using synthetic and real data.

Figures (8)

• Figure 1: Illustration of our training. Our method learns a neural binary occupancy field without off-surface labels. It uses the combination of a margin uncertainty sampling loss near the surface (Green), maximizing entropy at the input point cloud samples (Red), and minimizing entropy everywhere else (Blue).
• Figure 2: ShapeNet shapenet reconstructions from sparse noisy unoriented point clouds.
• Figure 3: Faust Bogo:CVPR:2014 reconstructions from sparse unoriented point clouds.
• Figure 4: CD1 distance to GT for reconstructions of shape Gargoyle of benchmark SRB williams2019deep from a sparse unoriented point cloud.
• Figure 5: 3D Scene zhou2013dense reconstructions from sparse unoriented point clouds.