AnyDepth: Depth Estimation Made Easy

TL;DR

AnyDepth tackles zero-shot monocular depth estimation with a lightweight, data-centric approach. By freezing a strong DINOv3 encoder and replacing the heavy DPT decoder with the Simple Depth Transformer, it achieves competitive accuracy using significantly fewer parameters and FLOPs. The authors introduce data filtering based on depth distribution and gradient continuity, along with a Spatial Detail Enhancer and a learnable DySample upsampler to preserve details. Across five benchmarks and real-world edge devices, AnyDepth demonstrates favorable efficiency-accuracy trade-offs, highlighting the practical value of balancing model design with data quality.

Abstract

Monocular depth estimation aims to recover the depth information of 3D scenes from 2D images. Recent work has made significant progress, but its reliance on large-scale datasets and complex decoders has limited its efficiency and generalization ability. In this paper, we propose a lightweight and data-centric framework for zero-shot monocular depth estimation. We first adopt DINOv3 as the visual encoder to obtain high-quality dense features. Secondly, to address the inherent drawbacks of the complex structure of the DPT, we design the Simple Depth Transformer (SDT), a compact transformer-based decoder. Compared to the DPT, it uses a single-path feature fusion and upsampling process to reduce the computational overhead of cross-scale feature fusion, achieving higher accuracy while reducing the number of parameters by approximately 85%-89%. Furthermore, we propose a quality-based filtering strategy to filter out harmful samples, thereby reducing dataset size while improving overall training quality. Extensive experiments on five benchmarks demonstrate that our framework surpasses the DPT in accuracy. This work highlights the importance of balancing model design and data quality for achieving efficient and generalizable zero-shot depth estimation. Code: https://github.com/AIGeeksGroup/AnyDepth. Website: https://aigeeksgroup.github.io/AnyDepth.

Figures (10)

• Figure 1: We present AnyDepth, a simple and efficient training framework for zero-shot monocular depth estimation, which achieves impressive performance across a variety of indoor and outdoor scenes.
• Figure 2: Comparison of the number of parameters (left) and computational complexity (right) of AnyDepth and DPT for different model sizes and input resolutions. Our method significantly reduces the number of model parameters and computational cost while maintaining competitive accuracy.
• Figure 3: Comparison of inference time between AnyDepth and DPT at different input resolutions. Our method consistently achieves lower latency, especially at higher resolutions.
• Figure 4: AnyDepth architecture overview. The input image is encoded into tokens by a frozen DINOv3 backbone network, then decoded by our lightweight SDT decoder. Tokens undergo only a single projection and weighted fusion. The Spatial Detail Enhancer (SDE) module ensures finer-grained predictions. The feature map is upsampled by an efficient and learnable upsampler dysample, and the depth is finally output by the head.
• Figure 5: Dataset quality across the Total Score, Depth Distribution Score, and Gradient Continuity Score (higher is better).