RaNeuS: Ray-adaptive Neural Surface Reconstruction

TL;DR

RaNeuS tackles detailed 3D surface reconstruction from multi-view imagery by integrating a differentiable radiance field with a signed distance field (SDF) and introducing ray-adaptive regularization. By per-ray weighting of the Eikonal constraint via $\lambda_r(r)$ and a geometric-bias term $\lambda_g$, the method relaxes the SDF regularization where rendering is weak and strengthens it where radiance is well-learned, enabling thin structures to be preserved. It builds on hash-encoded neural rendering to achieve efficient training and demonstrates state-of-the-art performance in novel view synthesis and competitive geometry reconstruction on datasets including Mip-NeRF 360, NeRF-synthetic, and DTU. The approach provides a practical pathway for accurate, high-fidelity surface reconstruction in unbounded scenes from RGB inputs, with detailed surfaces such as leaves, ropes, and textiles.

Abstract

Our objective is to leverage a differentiable radiance field \eg NeRF to reconstruct detailed 3D surfaces in addition to producing the standard novel view renderings. There have been related methods that perform such tasks, usually by utilizing a signed distance field (SDF). However, the state-of-the-art approaches still fail to correctly reconstruct the small-scale details, such as the leaves, ropes, and textile surfaces. Considering that different methods formulate and optimize the projection from SDF to radiance field with a globally constant Eikonal regularization, we improve with a ray-wise weighting factor to prioritize the rendering and zero-crossing surface fitting on top of establishing a perfect SDF. We propose to adaptively adjust the regularization on the signed distance field so that unsatisfying rendering rays won't enforce strong Eikonal regularization which is ineffective, and allow the gradients from regions with well-learned radiance to effectively back-propagated to the SDF. Consequently, balancing the two objectives in order to generate accurate and detailed surfaces. Additionally, concerning whether there is a geometric bias between the zero-crossing surface in SDF and rendering points in the radiance field, the projection becomes adjustable as well depending on different 3D locations during optimization. Our proposed \textit{RaNeuS} are extensively evaluated on both synthetic and real datasets, achieving state-of-the-art results on both novel view synthesis and geometric reconstruction.

Figures (5)

• Figure 1: Comparison of the Eikonal regularizer with adaptive factor $\lambda_\text{r}$ and $\lambda_\text{g}$ against constant Eikonal regularizer's weights on unbounded scene reconstruction in Mip-NeRF 360 dataset barron2022mip.
• Figure 2: Comparison of Neuralangelo trained with and without the proposed geometric bias factor $\lambda_\text{g}$ on top of the model trained with $\lambda_\text{r}$.
• Figure 3: Geometric reconstruction comparison evaluated on the Mip-NeRF 360 dataset barron2022mip.
• Figure 4: Comparison of our mesh to Neus 2 wang2022neus2, focusing on some important details on the bonsai dataset that our method was able to reconstruct while NeuS 2 missed.
• Figure 5: Our adaptive training makes the learned geometry robust against the shadows triggered by the movement of the light source.