MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources

TL;DR

Metric Anything presents a scalable pretraining paradigm for metric depth estimation by leveraging diverse, noisy 3D sources and a Sparse Metric Prompt interface to decouple spatial reasoning from sensor biases. A large-scale dataset (~20M image-depth pairs across >10k cameras) enables a clear data-driven scaling trend, with proximal 3D supervision distilled into a prompt-free student that achieves state-of-the-art results across monocular metric depth, camera intrinsics recovery, multi-view 3D reconstruction, and VLA planning. The approach generalizes to unseen sensors and environments, improves spatial reasoning in multimodal models, and opens a practical path toward robust real-world metric perception without task-specific engineering. The work is open-sourced and demonstrates that metric depth can benefit from modern foundation-model scaling laws just as relative depth and 2D vision have already shown.

Abstract

Scaling has powered recent advances in vision foundation models, yet extending this paradigm to metric depth estimation remains challenging due to heterogeneous sensor noise, camera-dependent biases, and metric ambiguity in noisy cross-source 3D data. We introduce Metric Anything, a simple and scalable pretraining framework that learns metric depth from noisy, diverse 3D sources without manually engineered prompts, camera-specific modeling, or task-specific architectures. Central to our approach is the Sparse Metric Prompt, created by randomly masking depth maps, which serves as a universal interface that decouples spatial reasoning from sensor and camera biases. Using about 20M image-depth pairs spanning reconstructed, captured, and rendered 3D data across 10000 camera models, we demonstrate-for the first time-a clear scaling trend in the metric depth track. The pretrained model excels at prompt-driven tasks such as depth completion, super-resolution and Radar-camera fusion, while its distilled prompt-free student achieves state-of-the-art results on monocular depth estimation, camera intrinsics recovery, single/multi-view metric 3D reconstruction, and VLA planning. We also show that using pretrained ViT of Metric Anything as a visual encoder significantly boosts Multimodal Large Language Model capabilities in spatial intelligence. These results show that metric depth estimation can benefit from the same scaling laws that drive modern foundation models, establishing a new path toward scalable and efficient real-world metric perception. We open-source MetricAnything at http://metric-anything.github.io/metric-anything-io/ to support community research.

Figures (23)

• Figure 1: Overview of Metric Anything. (I) We aggregate diverse open-source 3D data into per-pixel metric depth maps, forming a $\sim$20M image–depth dataset captured by over 10,000 cameras across heterogeneous scenes. (II) Sparse Metric Prompts, generated by randomly masking depth maps, provide a minimal interface that decouples spatial reasoning from sensor and camera biases, enabling metric depth learning from noisy, heterogeneous sources. (III) The pretrained model and its distilled prompt-free student generalize robustly across multiple downstream tasks, revealing a clear scaling trend and establishing a solid foundation for versatile, data-driven metric perception.
• Figure 2: Scaling and Generalization. MetricAnything exhibits a clear scaling trend and strong overall downstream performance.
• Figure 3: Percentile Depth Range Comparison from Seven Datasets (Real-world vs. Our Pseudo Labels).
• Figure 4: Skip-Connection in ViT-DPT Architecture.
• Figure 5: Visualization of Depth SR and Completion. Our method better recovers missing regions with improved structure.