AllWeatherNet:Unified Image Enhancement for Autonomous Driving under Adverse Weather and Lowlight-conditions

TL;DR

AllWeather-Net tackles perception deterioration under adverse weather and low-light by learning a unified image enhancement model that preserves structure while improving color, texture, and illumination. It employs a residual generator $I' = G(I_S) + I_S$ within a two-network adversarial framework, augmented by Scaled Illumination-aware Attention Mechanism (SIAM) and a Hierarchical Discrimination Framework operating on scene-, object-, and texture-level patches, plus a Ranked Adaptive Window Pairing strategy. The method yields improved image quality and significant gains in semantic segmentation on training domains (up to 5.3% mIoU) and unseen domains (up to 3.9% mIoU), demonstrating strong cross-condition robustness and generalization. The work provides a first unified solution for adverse weather and low-light enhancement tailored for autonomous driving perception, supported by ablation analyses and public code.

Abstract

Adverse conditions like snow, rain, nighttime, and fog, pose challenges for autonomous driving perception systems. Existing methods have limited effectiveness in improving essential computer vision tasks, such as semantic segmentation, and often focus on only one specific condition, such as removing rain or translating nighttime images into daytime ones. To address these limitations, we propose a method to improve the visual quality and clarity degraded by such adverse conditions. Our method, AllWeather-Net, utilizes a novel hierarchical architecture to enhance images across all adverse conditions. This architecture incorporates information at three semantic levels: scene, object, and texture, by discriminating patches at each level. Furthermore, we introduce a Scaled Illumination-aware Attention Mechanism (SIAM) that guides the learning towards road elements critical for autonomous driving perception. SIAM exhibits robustness, remaining unaffected by changes in weather conditions or environmental scenes. AllWeather-Net effectively transforms images into normal weather and daytime scenes, demonstrating superior image enhancement results and subsequently enhancing the performance of semantic segmentation, with up to a 5.3% improvement in mIoU in the trained domain. We also show our model's generalization ability by applying it to unseen domains without re-training, achieving up to 3.9% mIoU improvement. Code can be accessed at: https://github.com/Jumponthemoon/AllWeatherNet.

Figures (13)

• Figure 1: Given images captured under adverse conditions in (a), we propose a method that can effectively adjust color and texture, modify lighting and shadows, and remove weather effects within a unified model. This results in a visually appealing appearance that resembles normal, day-like weather conditions (b), thereby enhancing the robust performance of autonomous driving perception systems.
• Figure 2: (a) Original Image. The evaluation of image processing techniques for semantic segmentation under adverse conditions reveals the deficiencies of (b) weather effect removal Chen2022MultiWeatherRemoval, (c) pixel-level translation zhu2017unpaired, and (d) low-light enhancement ma2022toward. Images processed by these methods either fail to sufficiently enhance image quality or introduce artifacts, affecting semantic prediction accuracy. (e) Our method, AllWeather-Net, effectively enhances color and texture detail while preserving most of the original image information, achieving the best performance.
• Figure 3: Comparison of pixel-level translation and image enhancement process.
• Figure 4: Overview of AllWeather-Net architecture. SIAM: Scaled Illumination-aware Attention Mechanism. AllWeather-Net can enhance images across all adverse conditions (e.g., fog, snow, rain, nighttime) with the help of the proposed SIAM and Hierarchical Discrimination Framework.
• Figure 5: Scaled illumination-aware attention mechanism in the generator.