EvalMVX: A Unified Benchmarking for Neural 3D Reconstruction under Diverse Multiview Setups

TL;DR

EvalMVX is proposed, a real-world dataset containing $25$ objects, each captured with a polarized camera under $20$ varying views and light conditions including OLAT and natural illumination, leading to $8,500$ images, facilitating quantitative benchmarking of MVX methods simultaneously.

Abstract

Recent advancements in neural surface reconstruction have significantly enhanced 3D reconstruction. However, current real world datasets mainly focus on benchmarking multiview stereo (MVS) based on RGB inputs. Multiview photometric stereo (MVPS) and multiview shape from polarization (MVSfP), though indispensable on high-fidelity surface reconstruction and sparse inputs, have not been quantitatively assessed together with MVS. To determine the working range of different MVX (MVS, MVSfP, and MVPS) techniques, we propose EvalMVX, a real-world dataset containing $25$ objects, each captured with a polarized camera under $20$ varying views and $17$ light conditions including OLAT and natural illumination, leading to $8,500$ images. Each object includes aligned ground-truth 3D mesh, facilitating quantitative benchmarking of MVX methods simultaneously. Based on our EvalMVX, we evaluate $13$ MVX methods published in recent years, record the best-performing methods, and identify open problems under diverse geometric details and reflectance types. We hope EvalMVX and the benchmarking results can inspire future research on multiview 3D reconstruction.

Figures (7)

• Figure 1: Dataset overview. EvalMVX contains 25 objects (left), covering diverse shapes and materials. From top to bottom, the materials range from simple diffuse to complex metallic and reflective ones, while the complexity of object shapes increases from left to right. With aligned GT shapes (middle), we can quantitatively evaluate MVX methods on the same object (right) and reveal their performance on detailed shape recovery and robustness against complex material.
• Figure 2: Capture setup and data processing of EvalMVX. (Top) Our capture system contains a polarization camera and 16 evenly distributed LEDs for recording multiview and multi-light images. (Bottom) Polarization information can be extracted from the snapshot polarization image.
• Figure 3: Inaccurate surface normal inputs from Uni-MS-PS hardy2024uni due to complex reflectance influence the shape recovery from SuperNormal supernormal.
• Figure 4: The difference between the real-captured AoP map and azimuth map due to sensor noise degrades the performance of MVSfP methods.
• Figure 5: Best-performing method for each object in EvalMVX, with CD displayed in brackets. Summary results shown in left and right correspond to using 20 and 10 input views.