Cross360: 360° Monocular Depth Estimation via Cross Projections Across Scales

TL;DR

Cross360 addresses the core challenge of accurate, globally consistent 360° depth estimation by merging local, distortion-free tangent-patch information with global equirectangular representations through a cross-attention framework. The CPFA module aligns TP and ERP features across scales, enabling each patch to inherit global context, while the PFAA module progressively aggregates multi-scale information to refine depth predictions. The approach yields state-of-the-art results on both real-world incomplete ERP inputs and complete synthetic datasets, demonstrates robust boundary handling, and maintains efficiency with a modest number of tangent patches. This work advances panoramic depth estimation by explicitly coupling geometry-aware cross-projection alignment with scale-aware feature integration, offering practical benefits for immersive navigation, robotics, and virtual reality systems. The methods are backed by extensive experiments, ablations, and comparisons showing meaningful improvements over prior multi-projection fusion approaches.

Abstract

360° depth estimation is a challenging research problem due to the difficulty of finding a representation that both preserves global continuity and avoids distortion in spherical images. Existing methods attempt to leverage complementary information from multiple projections, but struggle with balancing global and local consistency. Their local patch features have limited global perception, and the combined global representation does not address discrepancies in feature extraction at the boundaries between patches. To address these issues, we propose Cross360, a novel cross-attention-based architecture integrating local and global information using less-distorted tangent patches along with equirectangular features. Our Cross Projection Feature Alignment module employs cross-attention to align local tangent projection features with the equirectangular projection's 360° field of view, ensuring each tangent projection patch is aware of the global context. Additionally, our Progressive Feature Aggregation with Attention module refines multi-scaled features progressively, enhancing depth estimation accuracy. Cross360 significantly outperforms existing methods across most benchmark datasets, especially those in which the entire 360° image is available, demonstrating its effectiveness in accurate and globally consistent depth estimation. The code and model are available at https://github.com/huangkun101230/Cross360.

Figures (7)

• Figure 1: Framework Overview: Cross360 estimates 360° depth through three main components: an ERP-based feature encoder, Cross Projection Feature Alignment (CPFA), and Progressive Feature Aggregation with Attention (PFAA). The CPFA module aligns TP and ERP features, which are then concatenated with skip-linked ERP features and passed to the decoder. The decoder generates features at each level, used for multi-scale supervised learning and passed to the next CPFA level. The PFAA module processes all decoder features to produce the final depth map. Note that, multi-scale depth predictions during the decoding stage are not shown for better visualization.
• Figure 2: The architecture of Cross Projection Feature Alignment module.
• Figure 3: The architecture of Progressive Feature Aggregation with Attention module.
• Figure 4: Real-world data input with the original TP sampling strategy
• Figure 5: Real-world data input with the specific TP sampling strategy