2S-UDF: A Novel Two-stage UDF Learning Method for Robust Non-watertight Model Reconstruction from Multi-view Images

TL;DR

This work tackles reconstructing non-watertight 3D models from multi-view images by learning unsigned distance fields (UDF) via a novel two-stage approach, 2S-UDF. Stage 1 uses a simple bounded density to obtain a coarse UDF, while Stage 2 abandons density learning and directly refines the UDF by shaping an occlusion-aware weight function through ray truncation, yielding unbiased rendering. The two-stage decoupling improves training stability and reconstruction quality, outperforming state-of-the-art UDF methods on open-models like DeepFashion3D and on watertight datasets such as DTU, without requiring masks. The approach advances practical non-watertight reconstruction with robust open-surface handling and stable optimization, demonstrated across diverse datasets.

Abstract

Recently, building on the foundation of neural radiance field, various techniques have emerged to learn unsigned distance fields (UDF) to reconstruct 3D non-watertight models from multi-view images. Yet, a central challenge in UDF-based volume rendering is formulating a proper way to convert unsigned distance values into volume density, ensuring that the resulting weight function remains unbiased and sensitive to occlusions. Falling short on these requirements often results in incorrect topology or large reconstruction errors in resulting models. This paper addresses this challenge by presenting a novel two-stage algorithm, 2S-UDF, for learning a high-quality UDF from multi-view images. Initially, the method applies an easily trainable density function that, while slightly biased and transparent, aids in coarse reconstruction. The subsequent stage then refines the geometry and appearance of the object to achieve a high-quality reconstruction by directly adjusting the weight function used in volume rendering to ensure that it is unbiased and occlusion-aware. Decoupling density and weight in two stages makes our training stable and robust, distinguishing our technique from existing UDF learning approaches. Evaluations on the DeepFashion3D, DTU, and BlendedMVS datasets validate the robustness and effectiveness of our proposed approach. In both quantitative metrics and visual quality, the results indicate our superior performance over other UDF learning techniques in reconstructing 3D non-watertight models from multi-view images. Our code is available at https://bitbucket.org/jkdeng/2sudf/.

Figures (14)

• Figure 1: We learn a UDF from multiview images for non-watertight model reconstruction. As illustrated in the cross sections of learned UDFs, our learned UDF approximates to the ground truth. In contrast, the learned UDF of NeuralUDF Long2023 is choppy leading to significant artifacts, e.g., unexpected pit. The learned UDF of NeUDF Liu2023NeUDF is almost closed struggling to generate open surface.
• Figure 2: An intuitive illustration of our ray cutting algorithm, best viewed in color and magnified. A ray shoots from left to right, approaching the boundary of the first surface, and going through another two surfaces (gray boxes). The violet solid line represents the UDF values along the ray; the orange dashed line represents the corresponding color weight.
• Figure 3: Visual comparisons on selected models of the DeepFashion3D Zhu2020 dataset. The surfaces produced by NeuS and VolSDF are closed watertight models, thereby post-processing is required to remove the unnecessary parts. NeAT can produce open models by learning an SDF and predicting which surfaces in the extracted meshes should be removed, but it needs mask for supervision. NeuralUDF can generate open surfaces, but struggles with textureless inputs, leading to double-layered regions and large reconstruction errors. NeUDF generally performs well, but its training is unstable and may stumble on less distinguished, darker models like LS-D0. In contrast, our 2S-UDF consistently delivers effective reconstructions of non-watertight models. See the supplementary material for additional results.
• Figure 4: Visualization of the learned UDFs on cross sections. Compared with the ground truth, our method can learn a UDFs that most closely resemble the ground truth, among our method, NeuralUDF, and NeUDF. NeAT is omitted in this visualization, because it learns SDFs in lieu of UDFs. Note that for LS-D0, NeUDF completely collapses without a reasonable UDF learned.
• Figure 5: Plots of the Chamfer distance throughout the training process. Our method consistently reduces CD across both stages. In contrast, NeuralUDF, which also adopts a two-stage learning strategy, exhibits instability and yields a fragmented output following the second stage. The first-stage output of NeuralUDF, however, contains double-layered regions as marked above. In this figure, both methods start their stage 2 training at 250k iterations.

Theorems & Definitions (1)

• Theorem 1