PMNI: Pose-free Multi-view Normal Integration for Reflective and Textureless Surface Reconstruction

TL;DR

Experimental results show that the PMNI (Pose-free Multi-view Normal Integration) method achieves state-of-the-art performance in the reconstruction of reflective surfaces, even without reliable initial camera poses.

Abstract

Reflective and textureless surfaces remain a challenge in multi-view 3D reconstruction. Both camera pose calibration and shape reconstruction often fail due to insufficient or unreliable cross-view visual features. To address these issues, we present PMNI (Pose-free Multi-view Normal Integration), a neural surface reconstruction method that incorporates rich geometric information by leveraging surface normal maps instead of RGB images. By enforcing geometric constraints from surface normals and multi-view shape consistency within a neural signed distance function (SDF) optimization framework, PMNI simultaneously recovers accurate camera poses and high-fidelity surface geometry. Experimental results on synthetic and real-world datasets show that our method achieves state-of-the-art performance in the reconstruction of reflective surfaces, even without reliable initial camera poses.

Figures (8)

• Figure 1: (Top row) Given multi-view surface normals of a reflective and textureless surface, our method jointly recovers a high-fidelity surface (middle row) and accurate camera poses (bottom row). The reconstructed shape is comparable to the results of supernormal, which uses calibrated poses.
• Figure 2: Illustration of shape and pose estimation for reflective and textureless objects based on RGB and surface normals.
• Figure 3: Summary of existing neural surface reconstruction methods categorized by their surface reflectance types and camera calibration settings. The input and surface representation for each method are labeled in brackets.
• Figure 4: Qualitative comparison between SuperNormal supernormal (abbreviated by SN) and ours on DiLiGenT-MV dmv. The camera poses for "SN noise" are slightly perturbed to simulate calibration noise. Our method accurately recovers camera poses, and the reconstruction is robust to pose calibration noise.
• Figure 5: Qualitative comparison with SuperNormal supernormal (abbreviated as "SN") using surface normal maps estimated by SDM-UniPS sdm and StableNormal stablenormal (abbreviated as "SDM" and "ST"), respectively. The top row visualizes the input surface normals and their angular error distributions.