Robust ADAS: Enhancing Robustness of Machine Learning-based Advanced Driver Assistance Systems for Adverse Weather

TL;DR

This paper tackles the vulnerability of ML-ADAS to adverse weather by introducing Weather UNet (WUNet), a denoising preprocessing DNN based on the UNet architecture that transforms adverse-weather images into clear-weather inputs for downstream detectors like YOLOv8n. By synthetic augmentation of the KITTI dataset (fog, rain, snow) and a crop-based inference strategy, the authors show substantial improvements in object detection performance under harsh conditions, notably increasing mAP from 4% to 70% in extreme fog when using WUNet preprocessing. The approach avoids the costly retraining of all downstream DNNs, preserves driver visualization, and demonstrates the feasibility of edge-efficient robustness through data augmentation, color-representation analysis (RGB vs HSV), and cost mitigation. Overall, the work provides a practical, scalable pathway to enhance the safety and reliability of ML-ADAS in real-world adverse weather scenarios, with clear directions for future expansion and efficiency optimizations.

Abstract

In the realm of deploying Machine Learning-based Advanced Driver Assistance Systems (ML-ADAS) into real-world scenarios, adverse weather conditions pose a significant challenge. Conventional ML models trained on clear weather data falter when faced with scenarios like extreme fog or heavy rain, potentially leading to accidents and safety hazards. This paper addresses this issue by proposing a novel approach: employing a Denoising Deep Neural Network as a preprocessing step to transform adverse weather images into clear weather images, thereby enhancing the robustness of ML-ADAS systems. The proposed method eliminates the need for retraining all subsequent Depp Neural Networks (DNN) in the ML-ADAS pipeline, thus saving computational resources and time. Moreover, it improves driver visualization, which is critical for safe navigation in adverse weather conditions. By leveraging the UNet architecture trained on an augmented KITTI dataset with synthetic adverse weather images, we develop the Weather UNet (WUNet) DNN to remove weather artifacts. Our study demonstrates substantial performance improvements in object detection with WUNet preprocessing under adverse weather conditions. Notably, in scenarios involving extreme fog, our proposed solution improves the mean Average Precision (mAP) score of the YOLOv8n from 4% to 70%.

Figures (10)

• Figure 1: (a) Retraining all subsequent DNNs in the ML-ADAS framework to adapt to domain shifts in weather is costly. (b) Developing a Denoising DNN as a preprocessing step before feeding input to the ML-ADAS DNNs avoids retraining and outputs clear weather images.
• Figure 2: Performance of the YOLOv8n object detector, trained on the KITTI dataset, in adverse weather conditions.
• Figure 3: Overview of the weather robustification methodology using the UNet for image enhancement under adverse weather conditions
• Figure 4: Overview of our cost mitigation scheme. Each image is divided into crops which the WUNet processes. The prediction crops are joined together to output a full image after inference.
• Figure 5: Images from the KITTI dataset extended with image augmentations for adverse weather conditions. These images correspond to the high adversity validation sets. From left to right, in each column we show the base KITTI image, and the extreme fog, extreme rain, and extreme snow synthetic images respectively.