PSDF: Prior-Driven Neural Implicit Surface Learning for Multi-view Reconstruction

TL;DR

PSDF tackles the persistent challenge of reconstructing high-fidelity geometry from uncontrolled multi-view imagery by integrating external geometric priors from a pretrained MVS network with internal priors derived from neural implicit surface learning. The method introduces a visibility-aware feature consistency loss and depth prior-assisted sampling to constrain geometry using external priors, complemented by internal prior-guided importance rendering to reduce biased surface rendering during training. Through a two-stage training regime, PSDF achieves state-of-the-art results on Tanks and Temples and strong performance on DTU, demonstrating robust geometry recovery in challenging scenes. The work signals a practical advance for neural implicit surface reconstruction, enabling finer details and more reliable reconstructions in real-world scenarios.

Abstract

Surface reconstruction has traditionally relied on the Multi-View Stereo (MVS)-based pipeline, which often suffers from noisy and incomplete geometry. This is due to that although MVS has been proven to be an effective way to recover the geometry of the scenes, especially for locally detailed areas with rich textures, it struggles to deal with areas with low texture and large variations of illumination where the photometric consistency is unreliable. Recently, Neural Implicit Surface Reconstruction (NISR) combines surface rendering and volume rendering techniques and bypasses the MVS as an intermediate step, which has emerged as a promising alternative to overcome the limitations of traditional pipelines. While NISR has shown impressive results on simple scenes, it remains challenging to recover delicate geometry from uncontrolled real-world scenes which is caused by its underconstrained optimization. To this end, the framework PSDF is proposed which resorts to external geometric priors from a pretrained MVS network and internal geometric priors inherent in the NISR model to facilitate high-quality neural implicit surface learning. Specifically, the visibility-aware feature consistency loss and depth prior-assisted sampling based on external geometric priors are introduced. These proposals provide powerfully geometric consistency constraints and aid in locating surface intersection points, thereby significantly improving the accuracy and delicate reconstruction of NISR. Meanwhile, the internal prior-guided importance rendering is presented to enhance the fidelity of the reconstructed surface mesh by mitigating the biased rendering issue in NISR. Extensive experiments on the Tanks and Temples dataset show that PSDF achieves state-of-the-art performance on complex uncontrolled scenes.

Figures (8)

• Figure 1: The overview of PSDF. The training of PSDF is divided into two stages: non-importance rendering and importance rendering. For non-importance rendering stage, the samples $\mathcal{T}$ are obtained by error-bounded sampling and depth prior-assisted sampling, and the training is supervised by the external geometric priors. For importance rendering, the internal priors derived from non-importance rendering stage are used to form importance samples $\mathcal{T}_{\textrm{IR}}$ for rendering.
• Figure 2: Qualitative comparison of surface meshes reconstructed by various methods on the Tanks and Temples dataset.
• Figure 3: Qualitative comparison of surface meshes reconstructed by various methods on the DTU dataset.
• Figure 4: Visualization comparison of surface meshes reconstructed by PSDF with and without mask cropping on Scan83 and Scan97.
• Figure 5: Visualization comparison of ablation study on "Meetingroom" and "Barn" of the Tanks and Temples dataset.