Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing autonomous vehicle safety in rain: a data-centric approach for clear vision

Mark A. Seferian, Jidong J. Yang

TL;DR

This work tackles rain-induced perception challenges for camera-based autonomous driving by adopting a data-centric deraining approach. It introduces two batching schemes, STSB and STRB, within a DCGAN-inspired encoder-decoder architecture that uses UNet-like skip connections to remove rain streaks and perform weather-style transfer on 256×256 images. STRB achieves the best deraining quality, preserves scene details, and substantially improves steering performance under rain, demonstrated by a strong correlation to clear conditions ($R^2=0.956$) and lower mean absolute steering errors. The study leverages CARLA-generated rainy/clear pairs to validate safety implications, while acknowledging simulation limitations and outlining future directions including diffusion models and real-world datasets to extend robustness and enable downstream tasks such as object detection.

Abstract

Autonomous vehicles face significant challenges in navigating adverse weather, particularly rain, due to the visual impairment of camera-based systems. In this study, we leveraged contemporary deep learning techniques to mitigate these challenges, aiming to develop a vision model that processes live vehicle camera feeds to eliminate rain-induced visual hindrances, yielding visuals closely resembling clear, rain-free scenes. Using the Car Learning to Act (CARLA) simulation environment, we generated a comprehensive dataset of clear and rainy images for model training and testing. In our model, we employed a classic encoder-decoder architecture with skip connections and concatenation operations. It was trained using novel batching schemes designed to effectively distinguish high-frequency rain patterns from low-frequency scene features across successive image frames. To evaluate the model performance, we integrated it with a steering module that processes front-view images as input. The results demonstrated notable improvements in steering accuracy, underscoring the model's potential to enhance navigation safety and reliability in rainy weather conditions.

Enhancing autonomous vehicle safety in rain: a data-centric approach for clear vision

TL;DR

This work tackles rain-induced perception challenges for camera-based autonomous driving by adopting a data-centric deraining approach. It introduces two batching schemes, STSB and STRB, within a DCGAN-inspired encoder-decoder architecture that uses UNet-like skip connections to remove rain streaks and perform weather-style transfer on 256×256 images. STRB achieves the best deraining quality, preserves scene details, and substantially improves steering performance under rain, demonstrated by a strong correlation to clear conditions () and lower mean absolute steering errors. The study leverages CARLA-generated rainy/clear pairs to validate safety implications, while acknowledging simulation limitations and outlining future directions including diffusion models and real-world datasets to extend robustness and enable downstream tasks such as object detection.

Abstract

Autonomous vehicles face significant challenges in navigating adverse weather, particularly rain, due to the visual impairment of camera-based systems. In this study, we leveraged contemporary deep learning techniques to mitigate these challenges, aiming to develop a vision model that processes live vehicle camera feeds to eliminate rain-induced visual hindrances, yielding visuals closely resembling clear, rain-free scenes. Using the Car Learning to Act (CARLA) simulation environment, we generated a comprehensive dataset of clear and rainy images for model training and testing. In our model, we employed a classic encoder-decoder architecture with skip connections and concatenation operations. It was trained using novel batching schemes designed to effectively distinguish high-frequency rain patterns from low-frequency scene features across successive image frames. To evaluate the model performance, we integrated it with a steering module that processes front-view images as input. The results demonstrated notable improvements in steering accuracy, underscoring the model's potential to enhance navigation safety and reliability in rainy weather conditions.
Paper Structure (14 sections, 16 figures, 2 tables)

This paper contains 14 sections, 16 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Deraining
  3. Data collection
  4. CARLA simulator
  5. Image data
  6. Steering angle data
  7. Data-centric approach
  8. Data preparation and batching schemes
  9. Model architecture
  10. Model training and evaluation
  11. Batching scheme performance
  12. Comparison of deraining results
  13. Steering performance
  14. Conclusions

Figures (16)

  • Figure 1: Visualization of heavy rain effect when compared to original CARLA rain effect and modified CARLA rain effect.
  • Figure 2: Aerial views of the CARLA maps used. Town01, 03, 04, 07, and 10 for training; Town02 for validation; and Town05 for testing.
  • Figure 3: Batching Scheme 1: Sequential in time and sequential in batch (STSB).
  • Figure 4: Batching Scheme 2: Sequential in time and random in batch (STRB).
  • Figure 5: Batching Scheme 3: Random in time and random in batch (RTRB).
  • ...and 11 more figures