Non-aligned supervision for Real Image Dehazing

TL;DR

This paper proposes an innovative dehazing framework that operates under non-aligned supervision, and introduces a self-attention network based on mean and variance for modeling real infinite airlight, using the dark channel prior as positional guidance.

Abstract

Removing haze from real-world images is challenging due to unpredictable weather conditions, resulting in the misalignment of hazy and clear image pairs. In this paper, we propose an innovative dehazing framework that operates under non-aligned supervision. This framework is grounded in the atmospheric scattering model, and consists of three interconnected networks: dehazing, airlight, and transmission networks. In particular, we explore a non-alignment scenario that a clear reference image, unaligned with the input hazy image, is utilized to supervise the dehazing network. To implement this, we present a multi-scale reference loss that compares the feature representations between the referred image and the dehazed output. Our scenario makes it easier to collect hazy/clear image pairs in real-world environments, even under conditions of misalignment and shift views. To showcase the effectiveness of our scenario, we have collected a new hazy dataset including 415 image pairs captured by mobile Phone in both rural and urban areas, called "Phone-Hazy". Furthermore, we introduce a self-attention network based on mean and variance for modeling real infinite airlight, using the dark channel prior as positional guidance. Additionally, a channel attention network is employed to estimate the three-channel transmission. Experimental results demonstrate the superior performance of our framework over existing state-of-the-art techniques in the real-world image dehazing task. Phone-Hazy and code will be available at https://fanjunkai1.github.io/projectpage/NSDNet/index.html.

Figures (17)

• Figure 1: Dehazing results on real-world images from CVPRws 2021 jo2021multi and our Phone-Hazy. Our method can generate much clearer results compared to the state-of-the-art methods, RefineNet zhao2021refinednet, D$^{4}$yang2022self, PSD chen2021psd, DAD shao2020domain, CDD-GAN chen2022unpaired and RIDCP wu2023ridcp.
• Figure 2: Overall pipeline of our non-aligned supervision framework grounded in the atmosphere scattering model for the real image dehazing. This framework comprises essential components: a generator network for the dehazing image $J$, a mvSA network for the airlight map $A_{\infty}$, and a channel attention network for the transmission map $t$. Our heart part is the non-aligned supervision setting to train the dehazing generator network by leveraging a clear and non-aligned reference image, conveniently obtainable from the same scene. Another important part is the mvSA network to effectively estimate the $A_{\infty}$ by using dark channel prior in real scenes. Note that our framework stands apart from conventional supervised dehazing models as it operates without the need for the aligned ground truths.
• Figure 3: Dehazing results on the real-world smoke dataset. Our method effectively eliminates smoke and produces images that closely resemble the non-aligned reference. The red box indicates a zoomed-in patch, allowing for a more precise comparison.
• Figure 4: Dehazing results on the real-world Phone-Hazy dataset. Our method is capable of eliminating haze and producing images that closely resemble the reference image, even if they are not perfectly aligned.
• Figure 5: Dehazing results on the real-world RTTS dataset. Our method effectively eliminates haze in distant scenes while also enhancing the restoration of finer details.