Toward Robust Neural Reconstruction from Sparse Point Sets

TL;DR

This work tackles unsupervised 3D reconstruction by learning a neural signed distance function (SDF) from sparse, noisy point clouds using distributionally robust optimization (DRO). By modeling the training distribution with Wasserstein (WDRO) and Sinkhorn (SDRO) uncertainty sets, the method regularizes SDF learning against challenging query distributions, distributing error more evenly across the shape. The approach builds on Neural Pull (NP) and extends it with DRO, achieving superior reconstructions on ShapeNet, Faust, and real-world scenes, with SDRO offering faster convergence due to entropic regularization. Overall, the results demonstrate that incorporating distributional robustness into neural implicit representations yields robust, high-fidelity shapes from highly imperfect inputs, suggesting broad applicability to real-world 3D sensing tasks.

Abstract

We consider the challenging problem of learning Signed Distance Functions (SDF) from sparse and noisy 3D point clouds. In contrast to recent methods that depend on smoothness priors, our method, rooted in a distributionally robust optimization (DRO) framework, incorporates a regularization term that leverages samples from the uncertainty regions of the model to improve the learned SDFs. Thanks to tractable dual formulations, we show that this framework enables a stable and efficient optimization of SDFs in the absence of ground truth supervision. Using a variety of synthetic and real data evaluations from different modalities, we show that our DRO based learning framework can improve SDF learning with respect to baselines and the state-of-the-art methods.

Figures (11)

• Figure 1: We fit a neural SDF to a sparse noisy point cloud solely, using a distributionally robust loss function. Compared to the state-of-the-art, our method provides more faithful and robust reconstructions, as can be seen in these detailed and thin structures of ShapeNet shapenet objects.
• Figure 2: We learn a neural SDF $f_{\theta}$ from a point cloud (black dots) by minimizing the error between projection of spatial queries $\{q\}$ on the level set of the field (gray curve) and their nearest input point $p$. Instead of learning with a standard predefined distribution of queries $Q$, we optimize for the worst-case query distribution $Q'$ within a ball of distributions around $Q$.
• Figure 3: Faust Bogo:CVPR:2014 reconstructions. CONet and POCO use data priors.
• Figure 4: 3D Scene zhou2013dense reconstructions from sparse unoriented point clouds.
• Figure 5: SemanticPOSS semantic reconstruction from road scene LiDAR data.