NightHaze: Nighttime Image Dehazing via Self-Prior Learning

TL;DR

The paper tackles nighttime image dehazing, where glow, low light, and noise create severe degradations and real paired data are scarce. It introduces NightHaze, an MAE-inspired framework that uses severe augmentation—blending clear images with glow-like light maps and noise—to learn strong background priors, formalized by the augmentation $I = W_b J + (1 - W_b) L + \epsilon$. To mitigate residual artifacts, a self-refinement module based on a NF-IQA-guided semi-supervised teacher-student framework refines the model using real-world hazy images via a gated knowledge transfer mechanism, with $\text{Score} = F_{IQA}(F(w^{t+1}_s, x^{uh}_i)) + F_{IQA}(F(w^t_s, x^{uh}_i))$. The authors evaluate on RealNightHaze (440 images) using non-reference IQA metrics (MUSIQ, TRES, etc.) and demonstrate substantial performance gains over state-of-the-art nighttime dehazing methods, achieving scores closer to those of clear nighttime images. By bridging the synthetic-real domain gap and leveraging MAE-like priors, NightHaze offers robust nighttime haze removal with practical impact for real-world nighttime imaging.

Abstract

Masked autoencoder (MAE) shows that severe augmentation during training produces robust representations for high-level tasks. This paper brings the MAE-like framework to nighttime image enhancement, demonstrating that severe augmentation during training produces strong network priors that are resilient to real-world night haze degradations. We propose a novel nighttime image dehazing method with self-prior learning. Our main novelty lies in the design of severe augmentation, which allows our model to learn robust priors. Unlike MAE that uses masking, we leverage two key challenging factors of nighttime images as augmentation: light effects and noise. During training, we intentionally degrade clear images by blending them with light effects as well as by adding noise, and subsequently restore the clear images. This enables our model to learn clear background priors. By increasing the noise values to approach as high as the pixel intensity values of the glow and light effect blended images, our augmentation becomes severe, resulting in stronger priors. While our self-prior learning is considerably effective in suppressing glow and revealing details of background scenes, in some cases, there are still some undesired artifacts that remain, particularly in the forms of over-suppression. To address these artifacts, we propose a self-refinement module based on the semi-supervised teacher-student framework. Our NightHaze, especially our MAE-like self-prior learning, shows that models trained with severe augmentation effectively improve the visibility of input haze images, approaching the clarity of clear nighttime images. Extensive experiments demonstrate that our NightHaze achieves state-of-the-art performance, outperforming existing nighttime image dehazing methods by a substantial margin of 15.5% for MUSIQ and 23.5% for ClipIQA.

Figures (7)

• Figure 1: Qualitative results from NightEnhance'23 jin2023enhancing, NightDeFog'20 yan2020nighttime, DiT'23 Peebles2023DiT and our method, on the real-world dataset. Ours not only suppress glow but also reveal the detailed textures of the night scenes, including those under low light and strong glow.
• Figure 2: Overview of our self-prior learning. This approach aims to produce strong priors that are resilient to real-world night haze degradations. Given a clear nighttime image, we blend clear images with various maps collected from real-world haze images and add noise to these blended images. The augmented input is then fed into an Encoder-Decoder framework to recover the clear background. The loss is a simple L1 loss between the reconstructed image and the clear input. The key to self-prior learning is the severity of the augmentation. By increasing the noise values to approach as high as the pixel intensity values of the glow and light effect blended images, our augmentation becomes severe, resulting in stronger priors.
• Figure 3: Example results on validation sets. For each triplet, we show the augmented image (left), our restoration (middle), and the ground-truth (right).
• Figure 4: Visualization of different light maps that contain glow effects. We use Gaussian kernels to simulate glow effects. By adjusting the parameters of kernels, we can control the number, size, and brightness of glow regions.
• Figure 5: Visualization of blending weight maps, where the black and white regions represent low and high blending values. By adjusting the values of blending weights and the noise term, we can control the severity of our augmentation.