PVP-Recon: Progressive View Planning via Warping Consistency for Sparse-View Surface Reconstruction

TL;DR

This work tackles sparse-view neural implicit surface reconstruction by pairing a warping-consistency based view-planning module with a progressive hash-encoded SDF reconstruction. The view planner selects the next best view to maximize information gain via cross-view image warping, enabling high-quality geometry with a small set of views. The reconstruction module employs a progressive activation scheme for multi-resolution hash features and introduces a directional Hessian loss to regularize the SDF near the surface, improving robustness under sparse views. Across three benchmarks and a robotics scenario, PVP-Recon outperforms baselines and demonstrates flexible integration with other reconstruction frameworks, highlighting its practical value for active and resource-efficient 3D sensing.

Abstract

Neural implicit representations have revolutionized dense multi-view surface reconstruction, yet their performance significantly diminishes with sparse input views. A few pioneering works have sought to tackle the challenge of sparse-view reconstruction by leveraging additional geometric priors or multi-scene generalizability. However, they are still hindered by the imperfect choice of input views, using images under empirically determined viewpoints to provide considerable overlap. We propose PVP-Recon, a novel and effective sparse-view surface reconstruction method that progressively plans the next best views to form an optimal set of sparse viewpoints for image capturing. PVP-Recon starts initial surface reconstruction with as few as 3 views and progressively adds new views which are determined based on a novel warping score that reflects the information gain of each newly added view. This progressive view planning progress is interleaved with a neural SDF-based reconstruction module that utilizes multi-resolution hash features, enhanced by a progressive training scheme and a directional Hessian loss. Quantitative and qualitative experiments on three benchmark datasets show that our framework achieves high-quality reconstruction with a constrained input budget and outperforms existing baselines.

Figures (12)

• Figure 1: Overview of our proposed system. The overall framework consists of two collaborative modules. In the reconstruction module, we use hash encoding to represent the SDF of object geometry, and adopt a training scheme that progressively enables finer levels of hash features. In the view planning module, we design a simple yet effective warping-based scoring strategy that progressively supplements the most informative views for surface reconstruction.
• Figure 2: Illustration of our view planning strategy on DTU scan114. For each candidate view, we render the current reconstructed mesh from this view. Then, we use the rendered depth and silhouette mask to warp the rendered RGB image into the closest training view and evaluate the warping score.
• Figure 3: Comparison of surface reconstruction on DTU and BlendedMVS datasets. Our method generates the most accurate and detailed results.
• Figure 4: Comparison of image rendering on Blender dataset. Our method can generate high-quality renderings with richer details.
• Figure 5: We show how the reconstructed mesh of BlendedMVS Sculpture changes as new input views are progressively added. Note that the newly added views provide beneficial information that makes the reconstruction result more precise and detailed.