Den-SOFT: Dense Space-Oriented Light Field DataseT for 6-DOF Immersive Experience

TL;DR

Den-SOFT tackles the lack of large-scale, dense light-field datasets for 6DoF VR by designing a mobile 46-camera rig and capturing 5K imagery across indoor and outdoor scenes, achieving an average density of $134.68$ viewpoints per unit sphere. The authors validate the dataset with three reconstruction paradigms—IBRNet, Instant-NGP, and 3D Gaussian Splatting—and demonstrate VR-ready rendering by integrating results into Unity. Nine scenes (7 outdoor, 2 indoor) reach 5K resolution with densities up to $189$ VP/m^3, illustrating the dataset's ability to cover foreground and background details for immersive experiences. This work lays a foundation for large-scale scene reconstruction research and points to future extensions in dynamic capture, semantic understanding, and real-time light-field pipelines.

Abstract

We have built a custom mobile multi-camera large-space dense light field capture system, which provides a series of high-quality and sufficiently dense light field images for various scenarios. Our aim is to contribute to the development of popular 3D scene reconstruction algorithms such as IBRnet, NeRF, and 3D Gaussian splitting. More importantly, the collected dataset, which is much denser than existing datasets, may also inspire space-oriented light field reconstruction, which is potentially different from object-centric 3D reconstruction, for immersive VR/AR experiences. We utilized a total of 40 GoPro 10 cameras, capturing images of 5k resolution. The number of photos captured for each scene is no less than 1000, and the average density (view number within a unit sphere) is 134.68. It is also worth noting that our system is capable of efficiently capturing large outdoor scenes. Addressing the current lack of large-space and dense light field datasets, we made efforts to include elements such as sky, reflections, lights and shadows that are of interest to researchers in the field of 3D reconstruction during the data capture process. Finally, we validated the effectiveness of our provided dataset on three popular algorithms and also integrated the reconstructed 3DGS results into the Unity engine, demonstrating the potential of utilizing our datasets to enhance the realism of virtual reality (VR) and create feasible interactive spaces. The dataset is available at our project website.

Figures (6)

• Figure 1: A glance of raw data, camera poses and capture density of our Den-SOFT datasets. We collected data from a total of nine scenarios and visualized the capture volume. The sampling density from blue to red indicates sparse to dense.
• Figure 2: Visualization of Light field reconstruction results of three paradigms method: IBRNet, Instant-NGP and 3D Gaussian Splatting.
• Figure 3: 3D Gaussian Splatting reconstruction results under different image density input.
• Figure 4: 3DGS performance under different capture density.
• Figure 5: Den-SOFT's potential application: users can import training results into game engines for a six-degrees-of-freedom immersive VR experience.