Domain Adaptation based Object Detection for Autonomous Driving in Foggy and Rainy Weather

TL;DR

This work tackles the domain shift in autonomous-driving object detection arising from foggy and rainy weather. It introduces an unsupervised domain-adaptive framework built on a Faster R-CNN backbone with Dynamic Masking, image- and object-level domain classifiers, an Adversarial Gradient Reversal Layer (AdvGRL) for hard-example mining, and a domain-level metric regularization using an auxiliary augmented domain with triplet losses across source, auxiliary, and target domains. The approach yields State-of-the-Art gains on Cityscapes-derived foggy and rainy benchmarks and demonstrates strong cross-camera transfer to KITTI, with ablations confirming the contribution of each component. The method offers practical benefits for robust perception in adverse weather, advancing safe and reliable autonomous driving in real-world conditions.

Abstract

Typically, object detection methods for autonomous driving that rely on supervised learning make the assumption of a consistent feature distribution between the training and testing data, this such assumption may fail in different weather conditions. Due to the domain gap, a detection model trained under clear weather may not perform well in foggy and rainy conditions. Overcoming detection bottlenecks in foggy and rainy weather is a real challenge for autonomous vehicles deployed in the wild. To bridge the domain gap and improve the performance of object detection in foggy and rainy weather, this paper presents a novel framework for domain-adaptive object detection. The adaptations at both the image-level and object-level are intended to minimize the differences in image style and object appearance between domains. Furthermore, in order to improve the model's performance on challenging examples, we introduce a novel adversarial gradient reversal layer that conducts adversarial mining on difficult instances in addition to domain adaptation. Additionally, we suggest generating an auxiliary domain through data augmentation to enforce a new domain-level metric regularization. Experimental findings on public benchmark exhibit a substantial enhancement in object detection specifically for foggy and rainy driving scenarios.

Figures (9)

• Figure 1: Illustration of the weather domain gap (foggy and rainy) for autonomous driving and the detection performance drop because of the domain gap. Three deep learning models ( Faster R-CNN ren2015faster, Mask R-CNN he2017mask, and YOLOv4 bochkovskiy2020yolov4) are all trained with the clear weather data of Cityscapes cordts2016cityscapes.
• Figure 2: The architecture of the proposed domain adaptation-based enhanced detection for intelligent vehicles in foggy and rainy weather. Here we illustrate our target domain using the example of foggy weather. It is recommended to view this figure in color.
• Figure 3: Illustration of the AdvGRL-based hard training example mining. We assign larger responses to harder training examples with lower domain classifier loss ($L_c$) values. In this paper, we set $\lambda_{0}=1$ and $\beta=30$.
• Figure 4: Sample visualization of Dynamic Masking Process (DMP): (a) the masked original image from Cityscapes cordts2016cityscapes, (b) masked synthesized foggy image, (c) masked synthesized rainy image.
• Figure 5: Illustration of synthesizing Rainy Cityscapes from the Cityscapes data: (a) the original image from Cityscapes cordts2016cityscapes, (b) rain map generated by RainMix guo2021efficientderain, (c) synthesized rainy image.