Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Always Clear Days: Degradation Type and Severity Aware All-In-One Adverse Weather Removal

Yu-Wei Chen, Soo-Chang Pei

TL;DR

A degradation type and severity aware model, called UtilityIR, for blind all-in-one bad weather image restoration that can outperform the state-of-the-art methods subjectively and objectively on different weather removal tasks with a large margin, and enjoy fewer model parameters.

Abstract

All-in-one adverse weather removal is an emerging topic on image restoration, which aims to restore multiple weather degradations in an unified model, and the challenge are twofold. First, discover and handle the property of multi-domain in target distribution formed by multiple weather conditions. Second, design efficient and effective operations for different degradations. To resolve this problem, most prior works focus on the multi-domain caused by different weather types. Inspired by inter\&intra-domain adaptation literature, we observe that not only weather type but also weather severity introduce multi-domain within each weather type domain, which is ignored by previous methods, and further limit their performance. To this end, we propose a degradation type and severity aware model, called UtilityIR, for blind all-in-one bad weather image restoration. To extract weather information from single image, we propose a novel Marginal Quality Ranking Loss (MQRL) and utilize Contrastive Loss (CL) to guide weather severity and type extraction, and leverage a bag of novel techniques such as Multi-Head Cross Attention (MHCA) and Local-Global Adaptive Instance Normalization (LG-AdaIN) to efficiently restore spatial varying weather degradation. The proposed method can outperform the state-of-the-art methods subjectively and objectively on different weather removal tasks with a large margin, and enjoy less model parameters. Proposed method even can restore unseen combined multiple degradation images, and modulate restoration level. Implementation code and pre-trained weights will be available at \url{https://github.com/fordevoted/UtilityIR}

Always Clear Days: Degradation Type and Severity Aware All-In-One Adverse Weather Removal

TL;DR

A degradation type and severity aware model, called UtilityIR, for blind all-in-one bad weather image restoration that can outperform the state-of-the-art methods subjectively and objectively on different weather removal tasks with a large margin, and enjoy fewer model parameters.

Abstract

All-in-one adverse weather removal is an emerging topic on image restoration, which aims to restore multiple weather degradations in an unified model, and the challenge are twofold. First, discover and handle the property of multi-domain in target distribution formed by multiple weather conditions. Second, design efficient and effective operations for different degradations. To resolve this problem, most prior works focus on the multi-domain caused by different weather types. Inspired by inter\&intra-domain adaptation literature, we observe that not only weather type but also weather severity introduce multi-domain within each weather type domain, which is ignored by previous methods, and further limit their performance. To this end, we propose a degradation type and severity aware model, called UtilityIR, for blind all-in-one bad weather image restoration. To extract weather information from single image, we propose a novel Marginal Quality Ranking Loss (MQRL) and utilize Contrastive Loss (CL) to guide weather severity and type extraction, and leverage a bag of novel techniques such as Multi-Head Cross Attention (MHCA) and Local-Global Adaptive Instance Normalization (LG-AdaIN) to efficiently restore spatial varying weather degradation. The proposed method can outperform the state-of-the-art methods subjectively and objectively on different weather removal tasks with a large margin, and enjoy less model parameters. Proposed method even can restore unseen combined multiple degradation images, and modulate restoration level. Implementation code and pre-trained weights will be available at \url{https://github.com/fordevoted/UtilityIR}
Paper Structure (27 sections, 11 equations, 19 figures, 4 tables)

This paper contains 27 sections, 11 equations, 19 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Single Image Restoration for Adverse Weather
  4. All-in-one Image Restoration
  5. Severity Aware Image Restoration
  6. Proposed Method
  7. Overview
  8. Network Architecture
  9. Degradation Information Encoder (DIE)
  10. RestoreNet
  11. Training Loss
  12. Experiments
  13. Implementation Details
  14. Comparisons with the State-of-the-art Methods
  15. Ablation Study
  16. ...and 12 more sections

Figures (19)

  • Figure 1: Illustration of challenge of handling various weather degraded images. Not only weather type result in multi-domain and different degradation appearance, weather severity also cause diverse visual appearance and inra-domain gap, which is ignored by previous all-in-one bad weather removal works.
  • Figure 2: (a) Overview of proposed UtilityIR. Input arbitrary weather image, Degradation Information Encoder (DIE) shown in (b) will first be applied to extract weather type and severity, then inject these information through Degradation Information Local-Global AdaIN (DI-LGAdaIN) in residual block shown in (c) and Degradation-guided Cross Attention (DGCA) shown in (d).
  • Figure 3: Illustration of insufficient of guided by standard MRL. The incorrect IQA interval prediction lead to apply inappropriate restoration level. Both cases achieve 0 in MRL.
  • Figure 4: Visual results of desnowing on Snow100K-L dataset. Zoom in for best view.
  • Figure 5: Visual results of deraindrop on Raindrop test-a dataset. Zoom in for best view.
  • ...and 14 more figures