ND-SDF: Learning Normal Deflection Fields for High-Fidelity Indoor Reconstruction

TL;DR

ND-SDF tackles high-fidelity indoor 3D reconstruction by learning a Normal Deflection Field that captures the angular deviation between true scene normals and prior normals using a quaternion-based deflection $\mathbf{Q}(\mathbf{r})$, with $\hat{\mathbf{N}}^d(\mathbf{r}) = \mathbf{Q}(\mathbf{r}) \otimes \hat{\mathbf{N}}(\mathbf{r}) \otimes \mathbf{Q}^{-1}(\mathbf{r})$. It introduces an adaptive deflection angle prior loss that weights high- vs low-frequency regions based on $\Delta\theta = \arccos( \hat{\mathbf{N}}(\mathbf{r}) \cdot \hat{\mathbf{N}}^d(\mathbf{r}) )$, and leverages three angle-guided optimizations—deflection-angle guided sampling, deflection-angle weighted photometric loss, and deflection-angle guided unbiased rendering—to recover thin and intricate structures while maintaining smooth surfaces. The approach uses multi-resolution hash grids, monocular priors (e.g., Omnidata), and a partial unbiasing scheme to stabilize convergence, achieving state-of-the-art results on ScanNet, ScanNet++, Replica, and Tanks & Temples, particularly excelling at thin geometries. Ablation studies validate the vital role of the deflection field, adaptive priors, and angle-guided strategies, and demonstrate robustness across different priors, highlighting practical impact for indoor scene understanding and related applications. Overall, ND-SDF provides a principled, geometry-aware framework for dynamic prior utilization in neural implicit reconstruction with strong generalization in challenging indoor environments.

Abstract

Neural implicit reconstruction via volume rendering has demonstrated its effectiveness in recovering dense 3D surfaces. However, it is non-trivial to simultaneously recover meticulous geometry and preserve smoothness across regions with differing characteristics. To address this issue, previous methods typically employ geometric priors, which are often constrained by the performance of the prior models. In this paper, we propose ND-SDF, which learns a Normal Deflection field to represent the angular deviation between the scene normal and the prior normal. Unlike previous methods that uniformly apply geometric priors on all samples, introducing significant bias in accuracy, our proposed normal deflection field dynamically learns and adapts the utilization of samples based on their specific characteristics, thereby improving both the accuracy and effectiveness of the model. Our method not only obtains smooth weakly textured regions such as walls and floors but also preserves the geometric details of complex structures. In addition, we introduce a novel ray sampling strategy based on the deflection angle to facilitate the unbiased rendering process, which significantly improves the quality and accuracy of intricate surfaces, especially on thin structures. Consistent improvements on various challenging datasets demonstrate the superiority of our method.

Figures (20)

• Figure 1: We present ND-SDF, a framework for high-fidelity 3D indoor surface reconstruction from multi-views. Shown above is an extracted mesh from ScanNet++.
• Figure 2: Overview of Our method. We utilize multi-resolution hash grids $\gamma_L$ as scene representation. The core of ND-SDF is the normal deflection field, which we represent using quaternions predicted by the deflection network (denoted as $f_d$). We align the deflected normals with the prior normals to learn the deviation between the scene and the priors. To distinctly supervise high and low-frequency areas, we employ an adaptive deflection angle prior loss, ensuring both smoothness and detail. Furthermore, we utilize the deflection angle $\Delta\theta$ to distinguish complex structures, enabling angle guided sampling and color loss to facilitate the recovery of intricate surface details. Lastly, we combine the unbiased rendering method zhang2023towards to ensure the generation of extremely thin structures indoors.
• Figure 3: We present a scenario in which a ray passes close to a chair leg. Along the ray, point A is situated near the chair leg, while point B is located at the ray-ground intersection. In the absence of unbiasing, abnormal density and rendering weight are observed at point A, whereas density should be exclusively assigned to point B.
• Figure 4: Qualitative results on ScanNet. Our approach effectively generates thin structures, such as chair legs, and produces more accurate surfaces.
• Figure 5: Visualization Results of the Models after Ablation on ScanNet++. Detailed definitions of these models and corresponding quantitative results are provided in Table \ref{['table:ablation_modules']}.