LUCES-MV: A Multi-View Dataset for Near-Field Point Light Source Photometric Stereo

TL;DR

LUCES-MV addresses the scarcity of near-field multi-view photometric stereo benchmarks by introducing a real-world dataset with 15 LEDs at 30–40 cm, 15 objects, ground-truth normals, meshes, and poses, plus full calibration data. It provides high-resolution near-field images and stereo pairs to enable end-to-end evaluation of PS methods from single-view to multi-view, including unposed cases. The paper benchmarks state-of-the-art single-view, binocular, and multi-view PS approaches, revealing that multi-view methods remain challenging on diverse, near-field objects and that the dataset offers a more difficult testbed than DiLiGenT-MV. The dataset is poised to drive progress toward robust, accurate, and scalable real-world PS-based 3D reconstruction methods.

Abstract

The biggest improvements in Photometric Stereo (PS) field has recently come from adoption of differentiable volumetric rendering techniques such as NeRF or Neural SDF achieving impressive reconstruction error of 0.2mm on DiLiGenT-MV benchmark. However, while there are sizeable datasets for environment lit objects such as Digital Twin Catalogue (DTS), there are only several small Photometric Stereo datasets which often lack challenging objects (simple, smooth, untextured) and practical, small form factor (near-field) light setup. To address this, we propose LUCES-MV, the first real-world, multi-view dataset designed for near-field point light source photometric stereo. Our dataset includes 15 objects with diverse materials, each imaged under varying light conditions from an array of 15 LEDs positioned 30 to 40 centimeters from the camera center. To facilitate transparent end-to-end evaluation, our dataset provides not only ground truth normals and ground truth object meshes and poses but also light and camera calibration images. We evaluate state-of-the-art near-field photometric stereo algorithms, highlighting their strengths and limitations across different material and shape complexities. LUCES-MV dataset offers an important benchmark for developing more robust, accurate and scalable real-world Photometric Stereo based 3D reconstruction methods.

Figures (13)

• Figure 1: This figure illustrates our capture setup (top half). Capture device is enclosed by the red casing which houses a 15 LED lights and two RGB cameras. The object (a miniature House in this case) is placed on a bluetooth controllable turntable covered by non-reflective material on the right. The average images from 15 lights of a particular view is shown on the top right corner. The ground truth mesh of the object scan is illustrated on the bottom of the figure. The shape of the House is far more complex than the shape of any object in current Photometric Stereo datasets such as DiLiGenT-MV LiZWSDT20. See examples of other challenging objects in Figure \ref{['fig:objects']}.
• Figure 2: In this figure we show a reconstruction result of Neuralangelo li2023neuralangelo using 36 pairs (average images of 15 light images are used) of stereo views. Note that the re-rendered images match the original very well (PSNR of 32.8 and 38.8 for Owl and Queen respectively). However, the predicted shape is significantly worse for Queen (1.68mm) than for Owl (0.44mm). This should be compared to results of RNb-NeuS BrumentRNb24 ( Owl - 0.37mm, Queen - 0.21mm), also shown qualitatively in Figure \ref{['fig:mainshape']}.
• Figure 3: The top part of figure shows iPhone images of 10 objects (first two rows) in the LUCES-MV dataset for which not only the photometric stereo images and ground truth meshes but also ground truth poses for 12 frames (6 pairs of stereo images) are provided. The final 5 objects (row 3) has all information except the alignment between images and ground truth meshes. We hope that future methods of unposed or weakly posed Photometric Stereo will be able to tackle the estimation and evaluation of the shape of these objects.
• Figure 4: This figure shows normal error map predictions for three normal estimation methods. Uni MS-PS hardyunips significantly outperforms other methods. See corresponding quantitative results in Table \ref{['tab:normalresults']}. Dark red color corresponds to an angluar error of 15 degrees and dark blue - to zero.
• Figure 5: This figure illustrates reconstruction to GT Hausdorff distance for each point of predicted meshes for all three MVPS methods. We emphasize that although the distance for all points is visualized, the numbers reported on Table \ref{['tab:mainresults']} are computed on visible points only (computing visibility for both GT and reconstructions). As indicated by Table \ref{['tab:mainresults']}, RNb-NeuS BrumentRNb24 shows same or slightly better performance than other two methods. In particular, significantly lower error on House object is very impressive and is likely due to implicit stereo matching of the texture through the use of albedo maps. As evident from the results shape error could be significantly improved on most of the objects. Improvements in both better normal estimation as well as neural rendering of pixel intensities are required to improve SOTA on LUCES-MV dataset. Dark red color corresponds to per point closest distance of 1 millimeter and dark blue - 0.