MetricDepth: Enhancing Monocular Depth Estimation with Deep Metric Learning

TL;DR

MetricDepth addresses the lack of natural class labels in monocular depth estimation by introducing differential-based sample identification, where depth differentials between anchor and samples guide positive/negative labeling. It augments deep metric learning with a multi-range strategy that partitions negative samples into depth-differential subgroups and applies subgroup-specific regularization, integrated with standard depth supervision via $L_{final}=L_{re}+L_{depth}$. Across NYU Depth V2 and KITTI, MetricDepth yields consistent improvements across diverse MDE models, demonstrating both indoor and outdoor robustness and the value of depth-aware feature regularization. The work demonstrates that mining depth-informed feature representations can meaningfully boost depth accuracy and detail, particularly for complex or thin structures, while highlighting avenues for adaptive hyperparameter tuning in future work.

Abstract

Deep metric learning aims to learn features relying on the consistency or divergence of class labels. However, in monocular depth estimation, the absence of a natural definition of class poses challenges in the leveraging of deep metric learning. Addressing this gap, this paper introduces MetricDepth, a novel method that integrates deep metric learning to enhance the performance of monocular depth estimation. To overcome the inapplicability of the class-based sample identification in previous deep metric learning methods to monocular depth estimation task, we design the differential-based sample identification. This innovative approach identifies feature samples as different sample types by their depth differentials relative to anchor, laying a foundation for feature regularizing in monocular depth estimation models. Building upon this advancement, we then address another critical problem caused by the vast range and the continuity of depth annotations in monocular depth estimation. The extensive and continuous annotations lead to the diverse differentials of negative samples to anchor feature, representing the varied impact of negative samples during feature regularizing. Recognizing the inadequacy of the uniform strategy in previous deep metric learning methods for handling negative samples in monocular depth estimation task, we propose the multi-range strategy. Through further distinction on negative samples according to depth differential ranges and implementation of diverse regularizing, our multi-range strategy facilitates differentiated regularization interactions between anchor feature and its negative samples. Experiments across various datasets and model types demonstrate the effectiveness and versatility of MetricDepth,confirming its potential for performance enhancement in monocular depth estimation task.

Figures (9)

• Figure 1: An illustration on the main difference between previous deep metric learning methods and our method. A denotes anchor, P denotes positive sample, and N denotes negative sample. In previous deep metric learning methods, it relies on class labels to distinguish sample types between different features. For monocular depth estimation task where no class labels exist, our method relies on depth differentials between features to identify sample types.
• Figure 2: An illustration on the core idea of our method. A denotes anchor, P denotes positive sample, and N denotes negative sample. Feature samples are identified as different types by referencing their depth differentials with anchor. For negative samples, the samples with diverse depth differentials are further distinguished and implemented with different regularizing formulas.
• Figure 3: An explanation on the feature sample collecting process in our method. In the diagram, the white lines divide the maps into different units. In (a), the original map is shifted by 2 units to the left and 1 unit to the top. The red boxes and the numbers denote different areas of the map, the green box denotes the anchor on a certain position, and the yellow box denotes a sample for the anchor. In (b), an entirely different map is shifted to current position acting as the sample for the original map.
• Figure 4: An example exposing the weakness of the uniform strategy on negative samples. (a), (b) and (c) show the input RGB image, the corresponding depth map, and the feature map of the RGB image. The white cross in (c) denotes the anchor feature. NS in the captions of (d), (e) and (f) means negative samples. In this example, with $r_{n}$ as 0.5, (d) shows all possible negative samples to the anchor feature. When $m_{u}$ is set as 2, (e) shows the negative samples which can produce non-zero loss values in Eq. \ref{['eq:bi_loss']}. (f) shows the ignored negative samples which produce no loss value and have no regularizing effect on the anchor feature according to Eq. \ref{['eq:bi_loss']}.
• Figure 5: Visualization examples of predicted depth maps from selected MDE models on NYU Depth V2.