LighthouseGS: Indoor Structure-aware 3D Gaussian Splatting for Panorama-Style Mobile Captures

TL;DR

LighthouseGS tackles indoor novel view synthesis from panorama-style mobile captures, where rotation-dominant motion and textureless regions hinder traditional SfM-based initialization. It introduces plane scaffold assembly to globally and locally align rough priors into a plane-aware 3D point set, followed by plane-guided initialization and stable pruning to robustly instantiate 3D Gaussians. A series of geometric and photometric corrections, including residual pose refinement and color correction, are integrated with differentiable rendering to recover accurate geometry and consistent appearance. The approach yields photorealistic renderings and enables practical AR applications such as object placement and panoramic view synthesis, validated on real and synthetic indoor datasets with ablations demonstrating the benefit of each component.

Abstract

We introduce LighthouseGS, a practical novel view synthesis framework based on 3D Gaussian Splatting that utilizes simple panorama-style captures from a single mobile device. While convenient, this rotation-dominant motion and narrow baseline make accurate camera pose and 3D point estimation challenging, especially in textureless indoor scenes. To address these challenges, LighthouseGS leverages rough geometric priors, such as mobile device camera poses and monocular depth estimation, and utilizes indoor planar structures. Specifically, we propose a new initialization method called plane scaffold assembly to generate consistent 3D points on these structures, followed by a stable pruning strategy to enhance geometry and optimization stability. Additionally, we present geometric and photometric corrections to resolve inconsistencies from motion drift and auto-exposure in mobile devices. Tested on real and synthetic indoor scenes, LighthouseGS delivers photorealistic rendering, outperforming state-of-the-art methods and enabling applications like panoramic view synthesis and object placement. Project page: https://vision3d-lab.github.io/lighthousegs/

Figures (8)

• Figure 1: Comparison between complex capturing motion and panorama-style motion for novel view synthesis.
• Figure 2: Overview of LighthouseGS. Given consecutive images captured by panorama-style motion with the corresponding rough geometric priors, we construct the plane scaffold that ensures global and local consistency. Then, we initialize 3D Gaussians to be aligned to scene geometry and optimize LighthouseGS with plane-aware stable optimization. To address motion drift and auto-exposure by the use of mobile devices, we additionally correct camera poses and view-dependent colors.
• Figure 3: Overview of plane scaffold assembly. Inputs are sequentially merged into the plane scaffold. The estimated depth from the current frame is globally-to-locally aligned with the projected global points from the previous set.
• Figure 4: Effect of plane-wise local alignment. Although the blue points of global alignment include local inconsistency, plane-wise local alignment ensures local consistency.
• Figure 5: Effect of stable pruning. Unlike previous pruning schemes, stable pruning keeps highly confident Gaussians, preserving precise scene geometry in textureless areas.