CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization

TL;DR

CAP-UDF tackles open and multi-layer surface reconstruction from raw point clouds by learning a consistent unsigned distance function $f$ and progressively guiding 3D queries toward the surface via the gradient $\nabla f$. The method introduces a consistency-aware field loss that stabilizes optimization and a progressive, stage-wise refinement that uses moved queries as priors to improve local detail. Surfaces are extracted directly from the learned UDF gradients with a gradient-based adaptation of marching cubes, avoiding post-processing like BPA. The framework extends to unsupervised point normal estimation and performs robustly across synthetic data, real scans, depth maps, and corrupted inputs, often surpassing state-of-the-art methods in accuracy and efficiency. This approach provides a practical, scalable solution for open-topology 3D reconstruction in diverse settings without requiring ground-truth distances or normals during training.

Abstract

Surface reconstruction for point clouds is an important task in 3D computer vision. Most of the latest methods resolve this problem by learning signed distance functions from point clouds, which are limited to reconstructing closed surfaces. Some other methods tried to represent open surfaces using unsigned distance functions (UDF) which are learned from ground truth distances. However, the learned UDF is hard to provide smooth distance fields due to the discontinuous character of point clouds. In this paper, we propose CAP-UDF, a novel method to learn consistency-aware UDF from raw point clouds. We achieve this by learning to move queries onto the surface with a field consistency constraint, where we also enable to progressively estimate a more accurate surface. Specifically, we train a neural network to gradually infer the relationship between queries and the approximated surface by searching for the moving target of queries in a dynamic way. Meanwhile, we introduce a polygonization algorithm to extract surfaces using the gradients of the learned UDF. We conduct comprehensive experiments in surface reconstruction for point clouds, real scans or depth maps, and further explore our performance in unsupervised point normal estimation, which demonstrate non-trivial improvements of CAP-UDF over the state-of-the-art methods.

Figures (23)

• Figure 1: Overview of CAP-UDF. Given a 3D query $q_i \in {Q_1}$ as input, the neural network predicts the unsigned distance $f(q_i)$ of $q_i$ and moves $q_i$ against the direction of gradient at $q_i$ with a stride of $f(q_i)$. The field consistency loss is then computed between the moved queries $q'_i$ and the target point cloud $P$ as the optimization target. After the network converges in the current stage, we update $P$ with a subset of $q'_i$ as additional priors to learn more local details in the next stage. Finally, we use the gradients of the learned UDFs in the field to model the relationship between different 3D grids and extract iso-surfaces.
• Figure 2: The level-sets show the distance fields learned by (a) Neural-Pull, (b) SAL, (c) NDF, and (d) Ours. The color of blue and red represent positive and negative distance, respectively. The darker the color, the closer it is to the approximated surface.
• Figure 3: Illustration of optimizing with different losses. (a) The initial distance field. (b) The distorted field with local minimum caused by inconsistent optimization with naive loss in Eq. (\ref{['eq:l2loss']}). (c) Optimizing with our consistency-aware loss in Eq. (\ref{['eq:gcloss']}) leads to correct and continuous distance field.
• Figure 4: Demonstration experiments on the effectiveness of consistency-aware loss. (a) The input point cloud of a double-deck wall. (b) The randomly sampled query points between two decks. (c, d) The moved queries optimized by naive loss in Eq. (\ref{['eq:l2loss']}) and our loss in Eq. (\ref{['eq:gcloss']}). (e,f) The learned distance field of a car with inner structure by the loss in Eq. (\ref{['eq:l2loss']}) and our loss in Eq. (\ref{['eq:gcloss']}).
• Figure 5: Illustration of progressively approximating the surface. (a) We sample queries on the high confidence (red) and low confidence (yellow) regions. (b, c) The optimization process in the first stage for learning UDFs by moving queries as Eq. (\ref{['eq:move']}). (d) We update the raw points with the moved queries in both regions as additional priors for learning more local details in the next stage. (e,f) More continuous and accurate field is achieved with the progressive learning schema.