OmniIndoor3D: Comprehensive Indoor 3D Reconstruction

TL;DR

OmniIndoor3D provides comprehensive 3D indoor scene understanding, which facilitates accurate and robust robotic navigation and achieves state-of-the-art results in appearance, geometry, and panoptic reconstruction.

Abstract

We propose a novel framework for comprehensive indoor 3D reconstruction using Gaussian representations, called OmniIndoor3D. This framework enables accurate appearance, geometry, and panoptic reconstruction of diverse indoor scenes captured by a consumer-level RGB-D camera. Since 3DGS is primarily optimized for photorealistic rendering, it lacks the precise geometry critical for high-quality panoptic reconstruction. Therefore, OmniIndoor3D first combines multiple RGB-D images to create a coarse 3D reconstruction, which is then used to initialize the 3D Gaussians and guide the 3DGS training. To decouple the optimization conflict between appearance and geometry, we introduce a lightweight MLP that adjusts the geometric properties of 3D Gaussians. The introduced lightweight MLP serves as a low-pass filter for geometry reconstruction and significantly reduces noise in indoor scenes. To improve the distribution of Gaussian primitives, we propose a densification strategy guided by panoptic priors to encourage smoothness on planar surfaces. Through the joint optimization of appearance, geometry, and panoptic reconstruction, OmniIndoor3D provides comprehensive 3D indoor scene understanding, which facilitates accurate and robust robotic navigation. We perform thorough evaluations across multiple datasets, and OmniIndoor3D achieves state-of-the-art results in appearance, geometry, and panoptic reconstruction. We believe our work bridges a critical gap in indoor 3D reconstruction. The code will be released at: https://ucwxb.github.io/OmniIndoor3D/

Figures (5)

• Figure 1: Comparison with existing methods. Unlike previous approaches that treat appearance, geometry, and panoptic reconstruction separately, our OmniIndoor3D presents a unified framework that leverages mutual dependencies for joint optimization, facilitating a comprehensive indoor 3D reconstruction essential for robotic navigation.
• Figure 2: Pipeline of OmniIndoor3D. Given posed RGB-D as inputs, we first extract a coarse 3D reconstruction to initialize the Gaussian distribution. The network subsequently optimizes three dedicated branches, each responsible for novel view synthesis, geometric reconstruction, and panoptic lifting.
• Figure 3: Visualization comparison for novel view synthesis.
• Figure 4: Visualization comparison for geometric reconstruction.
• Figure 5: Visualization comparison for panoptic lifting.