360-GeoGS: Geometrically Consistent Feed-Forward 3D Gaussian Splatting Reconstruction for 360 Images

TL;DR

360-GeoGS presents a feed-forward 360-degree 3D Gaussian Splatting pipeline that achieves geometrically consistent reconstruction by coupling Depth-Normal regularization with a depth prior derived from a spherical cost volume. The method fuses SphereCNN-based features, FiLM-modulated multi-scale representations, and a U-Net plus adapter to predict per-pixel Gaussian parameters, while enforcing surface coherence through the D-Normal loss. Empirical results on HM3D and Replica show strong improvements in geometric accuracy and depth consistency with rendering quality that remains competitive with state-of-the-art panoramic methods, validated by ablations that confirm each component’s contribution. The approach advances real-time, geometry-faithful 360 reconstruction for spatial perception tasks, with potential extensions to pose-free setups and occluded-region generation.

Abstract

3D scene reconstruction is fundamental for spatial intelligence applications such as AR, robotics, and digital twins. Traditional multi-view stereo struggles with sparse viewpoints or low-texture regions, while neural rendering approaches, though capable of producing high-quality results, require per-scene optimization and lack real-time efficiency. Explicit 3D Gaussian Splatting (3DGS) enables efficient rendering, but most feed-forward variants focus on visual quality rather than geometric consistency, limiting accurate surface reconstruction and overall reliability in spatial perception tasks. This paper presents a novel feed-forward 3DGS framework for 360 images, capable of generating geometrically consistent Gaussian primitives while maintaining high rendering quality. A Depth-Normal geometric regularization is introduced to couple rendered depth gradients with normal information, supervising Gaussian rotation, scale, and position to improve point cloud and surface accuracy. Experimental results show that the proposed method maintains high rendering quality while significantly improving geometric consistency, providing an effective solution for 3D reconstruction in spatial perception tasks.

Figures (5)

• Figure 1: Our pipeline extracts matching features from 360 images using a SphereCNN to construct a spherical cost volume and regress initial depth. Multi-scale features are also extracted by an image encoder and modulated via a FiLM module. The spherical cost volume, modulated multi-scale features, initial RGB images, and depth estimates are then fused to form a unified multi-modal representation, which is decoded by a U-Net and further refined by an adapter to produce per-pixel 3D Gaussian parameters. The network is trained with four losses: $\mathcal{L}_{\mathrm{rgb}}$, $\mathcal{L}_\mathrm{s}$, $\mathcal{L}_{\mathrm{dn}}$, and $\mathcal{L}_{\mathrm{depth}}$ (the definitions of $\mathcal{L}_\mathrm{s}$ and $\mathcal{L}_{\mathrm{dn}}$ are provided in Section \ref{['subsec:D-Normal']}).
• Figure 2: Predicted 3D Gaussian spatial distributions of the same scene reconstructed by different methods.
• Figure 3: Novel view rendering comparison of our method, Splatter-360, PanSplat, and MVSplat on the HM3D dataset.
• Figure 4: Novel view depth comparison among MVSplat, Splatter-360, and our method on the HM3D dataset.
• Figure 5: Illustration of the D-Normal regularization. $\bar{\mathbf{N}}_{d}$ is supervised by the ground-truth normal through $\mathcal{L}_{\mathrm{dn}}$(defined in subsection \ref{['sec:D-Normal formula']} ), guiding the flattened Gaussians to better fit the true surface.