Sparse Views, Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo

TL;DR

This work tackles the challenge of reconstructing high-fidelity 3D shapes from very sparse, uncalibrated multi-view imagery without requiring a dark room. It introduces an end-to-end differentiable framework that combines volume rendering (via VolSDF) with a physically grounded lighting model that blends ambient illumination and uncalibrated near-field point lights. Key contributions include the first multi-view uncalibrated point-light PS framework, a robust optimization strategy against cast-shadows, and an optimal diffuse albedo technique that yields strong performance in extremely sparse settings; results demonstrate superiority over state-of-the-art baselines on synthetic and real data, including textureless objects and wide baselines. Practically, this enables high-accuracy 3D capture outside laboratory environments with modest hardware and simple lighting, expanding the accessibility of photometric stereo in real-world applications.

Abstract

Neural approaches have shown a significant progress on camera-based reconstruction. But they require either a fairly dense sampling of the viewing sphere, or pre-training on an existing dataset, thereby limiting their generalizability. In contrast, photometric stereo (PS) approaches have shown great potential for achieving high-quality reconstruction under sparse viewpoints. Yet, they are impractical because they typically require tedious laboratory conditions, are restricted to dark rooms, and often multi-staged, making them subject to accumulated errors. To address these shortcomings, we propose an end-to-end uncalibrated multi-view PS framework for reconstructing high-resolution shapes acquired from sparse viewpoints in a real-world environment. We relax the dark room assumption, and allow a combination of static ambient lighting and dynamic near LED lighting, thereby enabling easy data capture outside the lab. Experimental validation confirms that it outperforms existing baseline approaches in the regime of sparse viewpoints by a large margin. This allows to bring high-accuracy 3D reconstruction from the dark room to the real world, while maintaining a reasonable data capture complexity.

Figures (13)

• Figure 1: We introduce the first framework for multi-view uncalibrated point-light photometric stereo. Given a set of PS images captured from different viewpoints (left), our method recovers high-fidelity $3$D reconstruction (right). The acquisition of uncalibrated point-light PS imagery captured under ambient lighting in a sparse multi-view setup does not only allow for easy data capture, but also leads to $3$D reconstructions of unprecedented detail. Here, with as few as two views we are able to reconstruct the squirrel's $3$D geometry at higher precision than the state-of-the-art.
• Figure 1: Results using three viewpoints with small camera baseline.
• Figure 2: Effect of cast-shadows on the result. The region behind the ear exhibits frequent cast-shadows in the input images (left), which induce artefacts in the estimated mesh (middle). Our truncation strategy (\ref{['sec:vanilla']}) successfully eliminates these (right).
• Figure 2: Vertex-to-mesh distance error maps. Errors are truncated for better visibility.
• Figure 3: Full 3D reconstruction of one synthetic and two real objects from 6 viewpoints. While some of the baseline reconstructions (with directional or point-light models) exhibit texture-induced patterns, distortions or other artefacts, the proposed framework consistently yields artefact- and distortion-free reconstructions with finer geometric details. Additional results are shown in the supplementary.