Geometric Prior-Guided Neural Implicit Surface Reconstruction in the Wild

TL;DR

This work tackles the challenge of reconstructing high-fidelity 3D surfaces from unconstrained image collections, such as internet photos of landmarks, by introducing explicit geometric priors into neural implicit surface optimization. The method, GeoNeucon-W, combines sparse SfM points with a displacement compensation strategy and robust normal priors (filtered by edge cues and multi-view consistency) to guide the SDF $f_{geo}$ and surface normals, yielding finer geometry than prior implicit-surface approaches. Extensive experiments on Heritage-Recon, Tanks and Temples, DTU, and custom/synthetic datasets demonstrate improved accuracy and detail in wild settings, highlighting its potential for digital preservation of cultural heritage and related applications. The approach advances open-world 3D reconstruction by embedding strong geometric constraints into neural representations, enabling more reliable geometry capture from diverse, uncontrolled photo collections while acknowledging trade-offs in computation and pose estimation reliability.

Abstract

Neural implicit surface reconstruction using volume rendering techniques has recently achieved significant advancements in creating high-fidelity surfaces from multiple 2D images. However, current methods primarily target scenes with consistent illumination and struggle to accurately reconstruct 3D geometry in uncontrolled environments with transient occlusions or varying appearances. While some neural radiance field (NeRF)-based variants can better manage photometric variations and transient objects in complex scenes, they are designed for novel view synthesis rather than precise surface reconstruction due to limited surface constraints. To overcome this limitation, we introduce a novel approach that applies multiple geometric constraints to the implicit surface optimization process, enabling more accurate reconstructions from unconstrained image collections. First, we utilize sparse 3D points from structure-from-motion (SfM) to refine the signed distance function estimation for the reconstructed surface, with a displacement compensation to accommodate noise in the sparse points. Additionally, we employ robust normal priors derived from a normal predictor, enhanced by edge prior filtering and multi-view consistency constraints, to improve alignment with the actual surface geometry. Extensive testing on the Heritage-Recon benchmark and other datasets has shown that the proposed method can accurately reconstruct surfaces from in-the-wild images, yielding geometries with superior accuracy and granularity compared to existing techniques. Our approach enables high-quality 3D reconstruction of various landmarks, making it applicable to diverse scenarios such as digital preservation of cultural heritage sites.

Figures (12)

• Figure 1: The proposed neural 3D surface reconstruction method aims to reconstruct high-fidelity 3D surfaces from unstructured internet photos of landmarks with varying appearances and complex occlusions. (a) Examples of input images of the Palacio de Bellas Artes. (b) Reconstructed 3D mesh of the landmark. (c) Detailed views showcasing the intricate geometric structures captured by our approach.
• Figure 2: The pipeline of the proposed method. COLMAP is used to generate sparse 3D points as priors to explicitly supervise the geometry network, and a displacement compensation is applied to mitigate noises in the sparse points. In addition, we utilize the normal priors generated by a pre-trained normal estimator to further impose geometric constraints on the object surface. To avoid ambiguity for the raw normal priors in some boundary regions with sharp edges, we employ edge priors and geometric consistency constraints to filter unreliable normals so that scene geometry can be optimized more accurately.
• Figure 3: Visualized semantic masks. These semantic masks are used to remove background and occluded areas.
• Figure 4: Geometric consistency constraint. We use multi-view consistency to remove the normal priors of those pixels that belong to occluded regions.
• Figure 5: Qualitative comparison between our method and NeuralRecon-W sun2022neural. As shown in the reconstructed meshes and corresponding local regions, the proposed method can better capture fine details.