OB3D: A New Dataset for Benchmarking Omnidirectional 3D Reconstruction Using Blender

TL;DR

OB3D introduces a Blender-generated omnidirectional dataset and benchmark tailored for 3D reconstruction, camera parameter estimation, and novel view synthesis. It provides 12 diverse scenes with ground-truth omnidirectional RGB, depth, normals, and sparse point clouds, along with exact camera parameters and evaluation protocols. The paper demonstrates baseline method performance across indoor/outdoor environments and egocentric vs non-egocentric trajectories, illustrating OB3D’s utility for fair, standardized comparisons and progress in omnidirectional reconstruction. By offering data on Kaggle and evaluation code on GitHub, OB3D aims to accelerate robust omnidirectional 3D methods and future real-world extensions.

Abstract

Recent advancements in radiance field rendering, exemplified by Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have significantly progressed 3D modeling and reconstruction. The use of multiple 360-degree omnidirectional images for these tasks is increasingly favored due to advantages in data acquisition and comprehensive scene capture. However, the inherent geometric distortions in common omnidirectional representations, such as equirectangular projection (particularly severe in polar regions and varying with latitude), pose substantial challenges to achieving high-fidelity 3D reconstructions. Current datasets, while valuable, often lack the specific focus, scene composition, and ground truth granularity required to systematically benchmark and drive progress in overcoming these omnidirectional-specific challenges. To address this critical gap, we introduce Omnidirectional Blender 3D (OB3D), a new synthetic dataset curated for advancing 3D reconstruction from multiple omnidirectional images. OB3D features diverse and complex 3D scenes generated from Blender 3D projects, with a deliberate emphasis on challenging scenarios. The dataset provides comprehensive ground truth, including omnidirectional RGB images, precise omnidirectional camera parameters, and pixel-aligned equirectangular maps for depth and normals, alongside evaluation metrics. By offering a controlled yet challenging environment, OB3Daims to facilitate the rigorous evaluation of existing methods and prompt the development of new techniques to enhance the accuracy and reliability of 3D reconstruction from omnidirectional images.

Figures (11)

• Figure 1: Examples of RGB image, depth map, normal map, and sparse 3D point cloud for indoor and outdoor scenes in OB3D.
• Figure 2: Pipeline of creating OB3D: RGB images (omnidirectional images), depth maps, normal maps, camera parameters for each image, and sparse 3D point cloud obtained using OpenMVG from a Bldender 3D project.
• Figure 3: Camera trajectories in OB3D: (i) Egocentric trajectory and (ii) Non-Egocentric trajectory.
• Figure 4: Data included in each scene of OB3D.
• Figure 5: Data included in each scene of OB3D (Continued).