Always Clear Depth: Robust Monocular Depth Estimation under Adverse Weather

TL;DR

ACDepth tackles robust monocular depth estimation under adverse weather by coupling diffusion-based data generation with LoRA-tuned translation, circular consistency, and adversarial training to create a high-quality degraded dataset. It introduces a multi-granularity knowledge distillation framework that merges supervision from a self-supervised teacher and a pretrained Depth Anything V2 model, enhanced by an ordinal guidance distillation module that concentrates learning on uncertain regions. The approach yields consistent improvements over prior methods on nuScenes and RobotCar, notably outperforming md4all-DD in night and rain scenarios, and is supported by extensive ablation analyses. This work demonstrates that high-fidelity synthetic degradation paired with diverse supervisory signals can substantially improve cross-domain generalization for depth perception in challenging real-world conditions.

Abstract

Monocular depth estimation is critical for applications such as autonomous driving and scene reconstruction. While existing methods perform well under normal scenarios, their performance declines in adverse weather, due to challenging domain shifts and difficulties in extracting scene information. To address this issue, we present a robust monocular depth estimation method called \textbf{ACDepth} from the perspective of high-quality training data generation and domain adaptation. Specifically, we introduce a one-step diffusion model for generating samples that simulate adverse weather conditions, constructing a multi-tuple degradation dataset during training. To ensure the quality of the generated degradation samples, we employ LoRA adapters to fine-tune the generation weights of diffusion model. Additionally, we integrate circular consistency loss and adversarial training to guarantee the fidelity and naturalness of the scene contents. Furthermore, we elaborate on a multi-granularity knowledge distillation strategy (MKD) that encourages the student network to absorb knowledge from both the teacher model and pretrained Depth Anything V2. This strategy guides the student model in learning degradation-agnostic scene information from various degradation inputs. In particular, we introduce an ordinal guidance distillation mechanism (OGD) that encourages the network to focus on uncertain regions through differential ranking, leading to a more precise depth estimation. Experimental results demonstrate that our ACDepth surpasses md4all-DD by 2.50\% for night scene and 2.61\% for rainy scene on the nuScenes dataset in terms of the absRel metric.

Figures (10)

• Figure 1: (a) Comparison of training data in challenging scenes: Compared to the data generation method used in md4all gasperini2023robust, the samples generated by our approach are more realistic, providing a better simulation of challenging real-world conditions. (b) Estimation results in challenging scenes: Our method consistently produces more accurate results than the existing md4all gasperini2023robust method, particularly in handling complex issues such as ground water reflections and nighttime object recognition.
• Figure 2: Overview of our ACDepth for robust monocular depth estimation.The teacher model is trained on simple samples using self-supervised learning, and the student model is trained on a mixed dataset of simple and complex samples using distillation learning. To provide the student model with supervisory signals beyond those from the teacher model, we designed a depth ranking loss $L_r$ leveraging ordinal information from the Depth Anything V2 model. To improve the student model's generalization across diverse scenarios, we incorporated a feature constraint loss $L_c$.
• Figure 3: Ordinal Pair Sampling: During training, we use two strategies to efficiently sample ordinal pairs. First, we employ the teacher and student models to respectively predict the depth maps (c and d) from the normal (a) and degraded (b) input, and then compute the pixel-wise errors (e) among them. Based on (e), we sample the local (uncertain regions with large errors) and global (random sampling regions) patches from the depth maps (d) and (f) to compute the total ordinal pair loss.
• Figure 4: Qualitative results on nuScenes caesar2020nuscenes and RobotCar maddern20171. We compare the ACDepth approach with md4all-DD and MonoDepth2, all of which use the same backbone. To better illustrate the results, the real point cloud is projected onto the original image, and no ground truth (GT) is required during training.
• Figure 5: Visualization of ablation study on the distillation learning, feature consistency constraint and ordinal guidance distillation.