SwiftNDC: Fast Neural Depth Correction for High-Fidelity 3D Reconstruction

TL;DR

SwiftNDC consistently reduces running time for accurate mesh reconstruction and boosts rendering fidelity for view synthesis, demonstrating the effectiveness of combining neural depth refinement with robust geometric initialization for high-fidelity and efficient 3D reconstruction.

Abstract

Depth-guided 3D reconstruction has gained popularity as a fast alternative to optimization-heavy approaches, yet existing methods still suffer from scale drift, multi-view inconsistencies, and the need for substantial refinement to achieve high-fidelity geometry. Here, we propose SwiftNDC, a fast and general framework built around a Neural Depth Correction field that produces cross-view consistent depth maps. From these refined depths, we generate a dense point cloud through back-projection and robust reprojection-error filtering, obtaining a clean and uniformly distributed geometric initialization for downstream reconstruction. This reliable dense geometry substantially accelerates 3D Gaussian Splatting (3DGS) for mesh reconstruction, enabling high-quality surfaces with significantly fewer optimization iterations. For novel-view synthesis, SwiftNDC can also improve 3DGS rendering quality, highlighting the benefits of strong geometric initialization. We conduct a comprehensive study across five datasets, including two for mesh reconstruction, as well as three for novel-view synthesis. SwiftNDC consistently reduces running time for accurate mesh reconstruction and boosts rendering fidelity for view synthesis, demonstrating the effectiveness of combining neural depth refinement with robust geometric initialization for high-fidelity and efficient 3D reconstruction.

Figures (5)

• Figure 1: Comparison of mesh reconstruction and novel view synthesis. Standard 3DGS based methods (Gaussian Opacity Field (GOF) yu2024gaussian) initialized from a sparse SfM point cloud requires extensive optimization to achieve high-fidelity results. SwiftNDC produces reliable dense geometry that accelerates mesh reconstruction (quality measured in Chamfer Distance (CD)$\downarrow$) and improves rendering quality for novel view synthesis.
• Figure 2: SwiftNDC pipeline. The method starts with both monocular and multi-view depth maps. A neural depth correction field, supervised by sparse SfM points, refines these depths at pixel level. The corrected depths are then transformed into a dense and reliable sampled point cloud, which serves as a strong geometric initialization for downstream mesh reconstruction and 3DGS-based view synthesis.
• Figure 3: Depth accuracy drives surface quality. The per-view–corrected monocular and multi-view depth maps look plausible, yet L1 distance comparison against GOF’s yu2024gaussian geo-consistent depth renderings reveals $\approx$40 mm residual error. Fusing these depths into point clouds exposes severe cross-view mis-alignment that propagates into rough, hole-ridden meshes with high Chamfer distance error. Our SwiftNDC cuts the depth error to less than 10 mm, producing a point cloud that aligns tightly across views and a final surface that is visually smooth and metrically accurate.
• Figure 4: Effect of removing unreliable points using reprojection-error filtering.
• Figure 5: Qualitative results for view synthesis and mesh reconstruction. Our dense initialization yields more complete geometry, especially in sparsely observed regions, enabling higher-quality rendering or reduced mesh reconstruction time with similar quality.