Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation

Lingdong Kong, Yaru Niu, Shaoyuan Xie, Hanjiang Hu, Lai Xing Ng, Benoit R. Cottereau, Liangjun Zhang, Hesheng Wang, Wei Tsang Ooi, Ruijie Zhu, Ziyang Song, Li Liu, Tianzhu Zhang, Jun Yu, Mohan Jing, Pengwei Li, Xiaohua Qi, Cheng Jin, Yingfeng Chen, Jie Hou, Jie Zhang, Zhen Kan, Qiang Ling, Liang Peng, Minglei Li, Di Xu, Changpeng Yang, Yuanqi Yao, Gang Wu, Jian Kuai, Xianming Liu, Junjun Jiang, Jiamian Huang, Baojun Li, Jiale Chen, Shuang Zhang, Sun Ao, Zhenyu Li, Runze Chen, Haiyong Luo, Fang Zhao, Jingze Yu

TL;DR

The RoboDepth Challenge investigates robust monocular depth estimation under out-of-distribution corruptions by leveraging KITTI-C and NYUDepth2-C benchmarks across two tracks (self-supervised outdoor and supervised indoor). It showcases diverse robust design strategies—from data augmentation (MAE mixing, CutFlip, APR, diffusion priors) and adversarial/structure-aware training to test-time restoration and diffusion-based priors—leading to multiple top-performing solutions. Key contributions include principled augmentation and learning techniques (IRUDepth, MAEMix, diffusion-based RDDepth, graph-based knowledge distillation, MRSF) and comprehensive OoD performance analyses, highlighting the importance of robustness, uncertainty, and model ensembling. The findings advance practical OoD depth estimation and point to future directions in foundation-model integration, broader robustness evaluation, and latency-aware deployments for safety-critical applications.

Abstract

Accurate depth estimation under out-of-distribution (OoD) scenarios, such as adverse weather conditions, sensor failure, and noise contamination, is desirable for safety-critical applications. Existing depth estimation systems, however, suffer inevitably from real-world corruptions and perturbations and are struggled to provide reliable depth predictions under such cases. In this paper, we summarize the winning solutions from the RoboDepth Challenge -- an academic competition designed to facilitate and advance robust OoD depth estimation. This challenge was developed based on the newly established KITTI-C and NYUDepth2-C benchmarks. We hosted two stand-alone tracks, with an emphasis on robust self-supervised and robust fully-supervised depth estimation, respectively. Out of more than two hundred participants, nine unique and top-performing solutions have appeared, with novel designs ranging from the following aspects: spatial- and frequency-domain augmentations, masked image modeling, image restoration and super-resolution, adversarial training, diffusion-based noise suppression, vision-language pre-training, learned model ensembling, and hierarchical feature enhancement. Extensive experimental analyses along with insightful observations are drawn to better understand the rationale behind each design. We hope this challenge could lay a solid foundation for future research on robust and reliable depth estimation and beyond. The datasets, competition toolkit, workshop recordings, and source code from the winning teams are publicly available on the challenge website.

The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation

TL;DR

The RoboDepth Challenge investigates robust monocular depth estimation under out-of-distribution corruptions by leveraging KITTI-C and NYUDepth2-C benchmarks across two tracks (self-supervised outdoor and supervised indoor). It showcases diverse robust design strategies—from data augmentation (MAE mixing, CutFlip, APR, diffusion priors) and adversarial/structure-aware training to test-time restoration and diffusion-based priors—leading to multiple top-performing solutions. Key contributions include principled augmentation and learning techniques (IRUDepth, MAEMix, diffusion-based RDDepth, graph-based knowledge distillation, MRSF) and comprehensive OoD performance analyses, highlighting the importance of robustness, uncertainty, and model ensembling. The findings advance practical OoD depth estimation and point to future directions in foundation-model integration, broader robustness evaluation, and latency-aware deployments for safety-critical applications.

Abstract

Accurate depth estimation under out-of-distribution (OoD) scenarios, such as adverse weather conditions, sensor failure, and noise contamination, is desirable for safety-critical applications. Existing depth estimation systems, however, suffer inevitably from real-world corruptions and perturbations and are struggled to provide reliable depth predictions under such cases. In this paper, we summarize the winning solutions from the RoboDepth Challenge -- an academic competition designed to facilitate and advance robust OoD depth estimation. This challenge was developed based on the newly established KITTI-C and NYUDepth2-C benchmarks. We hosted two stand-alone tracks, with an emphasis on robust self-supervised and robust fully-supervised depth estimation, respectively. Out of more than two hundred participants, nine unique and top-performing solutions have appeared, with novel designs ranging from the following aspects: spatial- and frequency-domain augmentations, masked image modeling, image restoration and super-resolution, adversarial training, diffusion-based noise suppression, vision-language pre-training, learned model ensembling, and hierarchical feature enhancement. Extensive experimental analyses along with insightful observations are drawn to better understand the rationale behind each design. We hope this challenge could lay a solid foundation for future research on robust and reliable depth estimation and beyond. The datasets, competition toolkit, workshop recordings, and source code from the winning teams are publicly available on the challenge website.
Paper Structure (67 sections, 32 equations, 34 figures, 24 tables)

This paper contains 67 sections, 32 equations, 34 figures, 24 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Depth Estimation
  4. Out-of-Distribution Perception
  5. Relevant Competitions
  6. Challenge Summary
  7. Overall Statistics
  8. Track # 1: Robust Self-Supervised Depth Estimation
  9. Track # 2: Robust Supervised Depth Estimation
  10. The RoboDepth Workshop
  11. Terms and Conditions
  12. Challenge Results
  13. Evaluation Metrics
  14. Track # 1 Results
  15. Track # 2 Results
  16. ...and 52 more sections

Figures (34)

  • Figure 1: The RoboDepth Challenge adopts the eighteen data corruption types from three main categories defined in the RoboDepth benchmark kong2023robodepth_benchmark. Examples shown are from the KITTI-C dataset.
  • Figure 2: We successfully hosted the RoboDepth Challenge at ICRA 2023.
  • Figure 3: The submission and scoring statistics for the two tracks in the RoboDepth Challenge.
  • Figure 4: Overview of the IRUDepth framework designed for robust self-supervised depth estimation.
  • Figure 5: Qualitative results of IRUDepth in the RoboDepth benchmark under different corruptions.
  • ...and 29 more figures