360Recon: An Accurate Reconstruction Method Based on Depth Fusion from 360 Images

TL;DR

360Recon introduces an ERP-aware multi-view reconstruction framework that tackles distortions in 360° panoramas via a Spherical Feature Extractor and Spherical Sweeping to build a discriminative 4D cost volume. A multi-scale Encoder–Decoder along with ERP-adapted 360° TSDF-style depth fusion yields high-precision depth maps and coherent 3D reconstructions on real-world datasets, without relying on heavy 3D CNNs. The approach achieves state-of-the-art depth estimation and 3D reconstruction across Matterport3D, Stanford2D3D, and OmniScenes, while maintaining efficient runtime and memory utilization. Limitations include occluded-region reconstruction, with future work planned around generative completion and sparse representations to improve completeness and efficiency.

Abstract

360-degree images offer a significantly wider field of view compared to traditional pinhole cameras, enabling sparse sampling and dense 3D reconstruction in low-texture environments. This makes them crucial for applications in VR, AR, and related fields. However, the inherent distortion caused by the wide field of view affects feature extraction and matching, leading to geometric consistency issues in subsequent multi-view reconstruction. In this work, we propose 360Recon, an innovative MVS algorithm for ERP images. The proposed spherical feature extraction module effectively mitigates distortion effects, and by combining the constructed 3D cost volume with multi-scale enhanced features from ERP images, our approach achieves high-precision scene reconstruction while preserving local geometric consistency. Experimental results demonstrate that 360Recon achieves state-of-the-art performance and high efficiency in depth estimation and 3D reconstruction on existing public panoramic reconstruction datasets.

Figures (10)

• Figure 1: Illustration of the proposed 360Recon algorithm, enabling precise 3D scene reconstruction through depth prediction from 360° images.
• Figure 1: Visualization of Depth Estimation on the Self-Collected Dataset. The first row displays panoramic color images, and the second row shows the corresponding depth estimation visualizations.
• Figure 2: Pipeline of our method: First, the spherical feature extractor mitigates distortion effects, followed by the enhancement of local geometric consistency using features extracted from ERP images. Finally, a self-developed 360° depth TSDF fusion approach is applied for the final 3D scene reconstruction.
• Figure 2: Visualization of 3D Reconstruction Results on the Self-Collected Dataset. We selected four images observed from different views of the reconstructed model using our self-collected dataset for display.
• Figure 3: Illustration of spherical sweeping: Based on multiple depth hypotheses, pixels from the reference frame are reprojected onto the source frame through spatial transformation to extract corresponding spherical features.