GeoSurDepth: Spatial Geometry-Consistent Self-Supervised Depth Estimation for Surround-View Cameras

TL;DR

GeoSurDepth addresses surround-view depth estimation by foregrounding geometric consistency as the primary supervisory cue in a self-supervised framework. It integrates foundation-model-based geometry priors (DepthAnything) and semantic guidance (CLIP) with a novel 2D-3D lifting view-synthesis pipeline and an adaptive motion-learning strategy to emphasize informative camera views. Key contributions include SNC for 3D surface normal coherence, DSC and SRC for edge-aware and cross-view depth consistency, and a spatial-dense-depth-based supervision that complements target-view reconstructions. Experiments on DDAD and nuScenes demonstrate state-of-the-art performance, validating geometry-coherence as a robust driver for multi-view depth estimation in autonomous driving contexts.

Abstract

Accurate surround-view depth estimation provides a competitive alternative to laser-based sensors and is essential for 3D scene understanding in autonomous driving. While prior studies have proposed various approaches that primarily focus on enforcing cross-view constraints at the photometric level, few explicitly exploit the rich geometric structure inherent in both monocular and surround-view setting. In this work, we propose GeoSurDepth, a framework that leverages geometry consistency as the primary cue for surround-view depth estimation. Concretely, we utilize foundation models as a pseudo geometry prior and feature representation enhancement tool to guide the network to maintain surface normal consistency in spatial 3D space and regularize object- and texture-consistent depth estimation in 2D. In addition, we introduce a novel view synthesis pipeline where 2D-3D lifting is achieved with dense depth reconstructed via spatial warping, encouraging additional photometric supervision across temporal, spatial, and spatial-temporal contexts, and compensating for the limitations of single-view image reconstruction. Finally, a newly-proposed adaptive joint motion learning strategy enables the network to adaptively emphasize informative spatial geometry cues for improved motion reasoning. Extensive experiments on DDAD and nuScenes demonstrate that GeoSurDepth achieves state-of-the-art performance, validating the effectiveness of our approach. Our framework highlights the importance of exploiting geometry coherence and consistency for robust self-supervised multi-view depth estimation.

Figures (8)

• Figure 1: Comparison of depth estimation performance between the proposed method GeoSurDepth and previous method CVCDepth.
• Figure 2: Network architecture of GeoSurDepth. Outputs of DA serve as surround-view geometry priors. Surround-view images at the target frame are first resized to $(518,518)$ before being fed into DA, and the output are interpolated back to original resolution. (a) Adaptive joint motion learning; (b) Cross-modal attention mechanism: For CLIP model, input images are resized to $(214,214)$ for token extraction.
• Figure 3: Illustration of spatial geometry priors-guided training.
• Figure 4: Visualization of depth estimation results on the DDAD (above) and nuScenes (below) datasets. White boxes indicate erroneous estimations. As observed, DepthAnything may also output inaccurate estimations in certain areas. We thus use it indirectly as pseudo priors.
• Figure 5: View synthesis example: (a) Color image; (b) Disparity map; (c) Reconstructed spatial dense depth; (d)(e) Spatial warping with estimated depth and reconstructed spatial dense depth.