From Transparent to Opaque: Rethinking Neural Implicit Surfaces with $α$-NeuS

TL;DR

This work extends NeuS to jointly reconstruct transparent and opaque surfaces by proving unbiased density mappings across the full opacity spectrum and introducing a DCUDF-based surface extraction pipeline. The method, named α-NeuS, relies on the local minima and zero iso-surface of the learned distance field to identify unbiased surfaces, and uses the absolute distance field along with DCUDF to robustly extract both surfaces. The authors validate on a 5 synthetic and 5 real-world dataset, showing improved completeness and surface fidelity over baselines like NeuS and NeUDF. The approach enables practical 3D reconstruction in scenes containing both thin transparent objects and opaque components, and provides public data and code for reproducibility and further research.

Abstract

Traditional 3D shape reconstruction techniques from multi-view images, such as structure from motion and multi-view stereo, face challenges in reconstructing transparent objects. Recent advances in neural radiance fields and its variants primarily address opaque or transparent objects, encountering difficulties to reconstruct both transparent and opaque objects simultaneously. This paper introduces $α$-Neus -- an extension of NeuS -- that proves NeuS is unbiased for materials from fully transparent to fully opaque. We find that transparent and opaque surfaces align with the non-negative local minima and the zero iso-surface, respectively, in the learned distance field of NeuS. Traditional iso-surfacing extraction algorithms, such as marching cubes, which rely on fixed iso-values, are ill-suited for such data. We develop a method to extract the transparent and opaque surface simultaneously based on DCUDF. To validate our approach, we construct a benchmark that includes both real-world and synthetic scenes, demonstrating its practical utility and effectiveness. Our data and code are publicly available at https://github.com/728388808/alpha-NeuS.

Figures (12)

• Figure 1: Conceptual illustration of the signed distances along a ray (black horizontal line) through a scene containing a single object (represented by a vertical line segment). (a) When the rendered opacity $\alpha$ is less than or equal to $0.5$, both the front and back faces of the object coincide with each other, aligning the maximal weight with the local minimum of the distance field. (b) When $\alpha$ exceeds $0.5$ but less than 1, the back face, which is not rendered, is separated from the front face. The further away the back face is, the more opaque the rendered front face is. The maximum weight in this case is aligned with the position of zero distance values. (c) For a fully opaque surface, the back face is infinitely away. The scene can therefore be considered the single ray-plane intersection discussed by NeuS Wang2021NeuS.
• Figure 2: Illustration of our mesh extraction procedure. (a) The orange line denotes the input model, where the dashed line is transparent and the solid line is opaque. The color map illustrates the distance field $f$. (b) The $r$ iso-curve (red) is extracted. (c) The iso-curve is mapped to the local minima of the absolute distance field $f^a$.
• Figure 3: Comparisons of projection on the mixed SDF and UDF $f$ and the absolute field $f^a$. The cutting plane draws the distance field. The white line indicates the 0 iso-surface and the orange line indicates the 0.005 iso-surface. (a) The extracted 0 iso-surface which attains the opaque surface exactly, but the transparent surface disappears. (b) The extracted 0.005 iso-surfaces. (c) Direct mapping on the original $f$ would result in the opaque surfaces shrinking. (d) In contrast, after taking the absolute, all unbiased surfaces are properly extracted.
• Figure 4: Qualitative comparison on synthetic data. Our method uses NeuS for distance field learning, and as shown in the normal maps, vanilla NeuS is in fact capable of reconstructing surfaces with transparency. The difference between our method and NeuS is drastic because NeuS cannot extract transparent surfaces where the distance field local minima are larger than zero with marching cubes, but our theory confirms and extends NeuS's learning ability, extracting both the non-negative local minima and the zero iso-surface.
• Figure 5: Percentage of sample points on the ground truth mesh that the distance to the reconstructed mesh is lower than given values. Blue: Ours, Orange: Zero iso-surface.

Theorems & Definitions (3)

• Theorem 1
• Remark
• proof