Rethinking Directional Parameterization in Neural Implicit Surface Reconstruction

TL;DR

This paper proposes a novel hybrid directional parameterization that is nearly parameter-free and can be effortlessly applied in any existing neural surface reconstruction method.

Abstract

Multi-view 3D surface reconstruction using neural implicit representations has made notable progress by modeling the geometry and view-dependent radiance fields within a unified framework. However, their effectiveness in reconstructing objects with specular or complex surfaces is typically biased by the directional parameterization used in their view-dependent radiance network. {\it Viewing direction} and {\it reflection direction} are the two most commonly used directional parameterizations but have their own limitations. Typically, utilizing the viewing direction usually struggles to correctly decouple the geometry and appearance of objects with highly specular surfaces, while using the reflection direction tends to yield overly smooth reconstructions for concave or complex structures. In this paper, we analyze their failed cases in detail and propose a novel hybrid directional parameterization to address their limitations in a unified form. Extensive experiments demonstrate the proposed hybrid directional parameterization consistently delivered satisfactory results in reconstructing objects with a wide variety of materials, geometry and appearance, whereas using other directional parameterizations faces challenges in reconstructing certain objects. Moreover, the proposed hybrid directional parameterization is nearly parameter-free and can be effortlessly applied in any existing neural surface reconstruction method.

Figures (8)

• Figure 1: Comparison of reconstruction results using different directional parameterizations in cai2023neuda. Using the viewing directional parameterization fails to reconstruct objects with specular surfaces, while using the reflection directional parameterization tends to result in incorrect geometry for objects with concave structures. Using our proposed hybrid parametrization shows consistently satisfactory reconstruction results compared to the existing parameterizations.
• Figure 2: Scenarios where existing directional parameterizations succeed and struggle. Reconstruction results are obtained by integrating the two directional parameterizations into NeuS wang2021neus.
• Figure 3: Impact of incorporating normals on view-dependent radiance modeling.(Left) Normals at sampled points may be influenced by unrelated surfaces other than the intersection surface. (Top right) Smooth surfaces yield similarity in normals and reflection directions for the sampled points, while (Bottom right) surfaces with intricate local details, e.g., concavities, induce a scattered distribution of normals and reflection directions (Ref. dirs), particularly pronounced slightly distant from the vicinity of the zero-level set, which adversely affects geometry optimization. The details are explained in Sec. \ref{['subsec:3_2']}.
• Figure 4: Qualitative comparison of integrating different directional parameterizations in NeuS wang2021neus and NeuDA cai2023neuda on the DTU dataset aanaes2016large and Shiny Blender dataset verbin2022ref.
• Figure 5: Qualitative comparison of integrating different directional parameterizations in NeuS wang2021neus and NeuDA cai2023neuda on the "toycar" scene from the real captured dataset hedman2021baking.