Seeing A 3D World in A Grain of Sand

TL;DR

Addressing miniature-scene 3D reconstruction, the paper introduces a snapshot, full-surround imaging approach using a planar catadioptric lens with eight mirror pairs on two nested octagonal pyramids. It maps the eight captured sub-images to virtual cameras and performs 3D Gaussian Splatting with a visual hull based depth constraint to handle sparse views. The method achieves state-of-the-art results on synthetic and real miniature scenes, outperforming existing sparse-view 3DGS baselines in both quality metrics and visual fidelity. The work enables robust, single-shot 360-degree miniature-scene reconstruction and has potential for dynamic scene capture with temporal consistency.

Abstract

We present a snapshot imaging technique for recovering 3D surrounding views of miniature scenes. Due to their intricacy, miniature scenes with objects sized in millimeters are difficult to reconstruct, yet miniatures are common in life and their 3D digitalization is desirable. We design a catadioptric imaging system with a single camera and eight pairs of planar mirrors for snapshot 3D reconstruction from a dollhouse perspective. We place paired mirrors on nested pyramid surfaces for capturing surrounding multi-view images in a single shot. Our mirror design is customizable based on the size of the scene for optimized view coverage. We use the 3D Gaussian Splatting (3DGS) representation for scene reconstruction and novel view synthesis. We overcome the challenge posed by our sparse view input by integrating visual hull-derived depth constraint. Our method demonstrates state-of-the-art performance on a variety of synthetic and real miniature scenes.

Figures (17)

• Figure 1: (a) A miniature scene with tiny object; (b) Example image captured by our catadioptric imaging system; (c) Novel view synthesis results.
• Figure 2: Left: A schematic illustration of our imaging system; Right: Sample image captured by the system.
• Figure 3: We map mirror reflections to virtual camera views by unfolding the light paths.
• Figure 4: The reflection light path of our mirror pair and the optimal coverage of a scene by our effective viewing volume.
• Figure 5: Our overall algorithmic pipeline.