Depth Anything with Any Prior

TL;DR

Prior Depth Anything addresses the problem of producing dense, metric-depth maps from monocular imagery when only incomplete and diverse depth priors are available. It introduces a coarse-to-fine pipeline that first performs pixel-level metric alignment to pre-fill priors using a frozen depth predictor, then refines the result with a conditioned monocular depth estimation model that jointly exploits the pre-filled prior and the predictor’s geometry under RGB guidance. Key contributions include pixel-level alignment with distance-aware weighting, a geometry- and metric-conditioned MDE, scale normalization to enable test-time MDE switching, and synthetic training data to simulate real-world priors. The approach achieves strong zero-shot performance across depth completion, super-resolution, and inpainting on seven datasets, including challenging mixed-prior scenarios, and provides a flexible accuracy-efficiency trade-off through model switching at test time, advancing practical deployment of depth estimation in diverse environments.

Abstract

This work presents Prior Depth Anything, a framework that combines incomplete but precise metric information in depth measurement with relative but complete geometric structures in depth prediction, generating accurate, dense, and detailed metric depth maps for any scene. To this end, we design a coarse-to-fine pipeline to progressively integrate the two complementary depth sources. First, we introduce pixel-level metric alignment and distance-aware weighting to pre-fill diverse metric priors by explicitly using depth prediction. It effectively narrows the domain gap between prior patterns, enhancing generalization across varying scenarios. Second, we develop a conditioned monocular depth estimation (MDE) model to refine the inherent noise of depth priors. By conditioning on the normalized pre-filled prior and prediction, the model further implicitly merges the two complementary depth sources. Our model showcases impressive zero-shot generalization across depth completion, super-resolution, and inpainting over 7 real-world datasets, matching or even surpassing previous task-specific methods. More importantly, it performs well on challenging, unseen mixed priors and enables test-time improvements by switching prediction models, providing a flexible accuracy-efficiency trade-off while evolving with advancements in MDE models.

Figures (11)

• Figure 1: Core Motivation. We progressively integrate complementary information from metric measurements (accurate metrics) and relative predictions (completeness and fine details) to produce dense and fine-grained metric depth maps.
• Figure 2: Prior Depth Anything. Considering RGB images, any form of depth prior $\mathbf{D}_\textrm{prior}$, and relative prediction $\mathbf{D}_\textrm{pred}$ from a frozen MDE model, coarse metric alignment first explicitly combines the metric data in $\mathbf{D}_\textrm{prior}$ and geometry structure in $\mathbf{D}_\textrm{pred}$ to fill the incomplete areas in $\mathbf{D}_\textrm{prior}$. Fine structure refinement implicitly merges the complementary information to produce the final metric depth map.
• Figure 3: Qualitative comparisons with previous methods. The depth prior or error map is shown below each sample.
• Figure 4: Error analysis on widely used but indeed noisy benchmarks silberman2012indoordai2017scannet. Red means higher error, while blue indicates lower error.
• Figure 5: Error analysis on RGB-D-D.