RealX3D: A Physically-Degraded 3D Benchmark for Multi-view Visual Restoration and Reconstruction

TL;DR

RealX3D introduces a real-capture benchmark for multi-view 3D restoration and reconstruction under nine real-world degradations across four families, with pixel-aligned LQ/GT pairs, RAW sensor data, and dense laser scans for metric depth and meshes. It provides a unified acquisition and processing pipeline that preserves high-resolution imagery, per-view depth, and world-scale geometry, enabling rigorous photometric and geometric evaluations. The dataset exposes significant gaps in current methods, showing that both optimization-based and feed-forward approaches struggle under real degradations such as low light, scattering, occlusion, and blur, while some pose estimation methods remain robust in challenging conditions. Overall, RealX3D offers a comprehensive, high-fidelity platform to benchmark and drive robustness improvements in 3D reconstruction and novel view synthesis for real-world scenarios.

Abstract

We introduce RealX3D, a real-capture benchmark for multi-view visual restoration and 3D reconstruction under diverse physical degradations. RealX3D groups corruptions into four families, including illumination, scattering, occlusion, and blurring, and captures each at multiple severity levels using a unified acquisition protocol that yields pixel-aligned LQ/GT views. Each scene includes high-resolution capture, RAW images, and dense laser scans, from which we derive world-scale meshes and metric depth. Benchmarking a broad range of optimization-based and feed-forward methods shows substantial degradation in reconstruction quality under physical corruptions, underscoring the fragility of current multi-view pipelines in real-world challenging environments.

Figures (5)

• Figure 1: RealX3D is a real-capture benchmark for 3D reconstruction under real-world degradations. It spans four general types across nine levels, including motion and defocus blur, lowlight, varying exposure, smoke, dynamic occlusion, and reflection. Each scene provides pixel-aligned low-quality and reference views, RAW data, and dense laser-scan geometry for comprehensive evaluation.
• Figure 2: Visualization of example meshes reconstructed from dense scanned point clouds.
• Figure 3: Overview of our data acquisition and processing pipeline: we calibrate cameras; set up a rail-dolly and studio lighting to capture pixel-aligned LQ/GT pairs; scan the scene and register camera poses to world coordinates; then back-project to recover per-view metric depth, and reconstruct a high-quality mesh from the scans.
• Figure 4: Qualitative reconstruction comparisons on RealX3D under selected six real-world degradations. We compare degraded inputs, baseline outputs, and GT, highlighting defects such as color bias, loss of detail, occlusion artifacts, and blur-induced oversmoothing.
• Figure 5: Visualizations of point clouds predicted by feed-forward foundation models on smoke and low-light scenes. For low-light scenes, the point cloud brightness is adjusted for better visibility.