LMHaze: Intensity-aware Image Dehazing with a Large-scale Multi-intensity Real Haze Dataset

TL;DR

This work presents LMHaze, a large-scale, high-quality real-world dataset that contains over 5K high-resolution image pairs and proposes a mixture-of-experts model based on Mamba (MoE-Mamba) for dehazing, which dynamically adjusts the model parameters according to the haze intensity.

Abstract

Image dehazing has drawn a significant attention in recent years. Learning-based methods usually require paired hazy and corresponding ground truth (haze-free) images for training. However, it is difficult to collect real-world image pairs, which prevents developments of existing methods. Although several works partially alleviate this issue by using synthetic datasets or small-scale real datasets. The haze intensity distribution bias and scene homogeneity in existing datasets limit the generalization ability of these methods, particularly when encountering images with previously unseen haze intensities. In this work, we present LMHaze, a large-scale, high-quality real-world dataset. LMHaze comprises paired hazy and haze-free images captured in diverse indoor and outdoor environments, spanning multiple scenarios and haze intensities. It contains over 5K high-resolution image pairs, surpassing the size of the biggest existing real-world dehazing dataset by over 25 times. Meanwhile, to better handle images with different haze intensities, we propose a mixture-of-experts model based on Mamba (MoE-Mamba) for dehazing, which dynamically adjusts the model parameters according to the haze intensity. Moreover, with our proposed dataset, we conduct a new large multimodal model (LMM)-based benchmark study to simulate human perception for evaluating dehazed images. Experiments demonstrate that LMHaze dataset improves the dehazing performance in real scenarios and our dehazing method provides better results compared to state-of-the-art methods.

Figures (4)

• Figure 1: Examples of our LMHaze dataset. It includs 5,040 pairs of real-world hazy and haze-free images, spanning both indoor and outdoor scenarios. Each column illustrates a specific scenario, presenting the haze-free image alongside its associated hazy counterparts with varying haze intensities from top to bottom.
• Figure 2: Overview of our MoE-Mamba framework. Given a hazy image $\hbox{\bf I}_{\text{h}}$, our goal is to restore a clean image $\hbox{\bf I}_{\text{dh}}$. We first feed prompt and $\hbox{\bf I}_{\text{h}}$ to the large multimodal model (LMM) Q-Align, and outputting the image degradation prior $\hat{\hbox{\bf p}}$. Meanwile, features $\hbox{\bf F}_\mathrm{s}$ are extracted by a 3$\times$3 conv layer. Then, the $\hat{\hbox{\bf p}}$ and $\hbox{\bf F}_\mathrm{s}$ are passed to multiple MoE-SSM (MM) Blocks to get refined feature $\hat{\hbox{\bf F}}_\mathrm{s}$. Then, element-wise addition is used to get $\hbox{\bf F}_\mathrm{a}$. The $\hbox{\bf I}_{\text{dh}}$ are reconstructed based on features $\hbox{\bf F}_\mathrm{a}$. The key elements of our MM Block, i.e., the Mixture-of-Experts (MoE) Block and the State Space Model (SSM) Block, are shown in the blue and the green dotted box.
• Figure 3: Example results on LMHaze dataset. (a)-(b) are the hazy image and corresponding haze-free image. (c)-(g) show the dehazing results of different SOTA dehazing models. Our model achieves the best dehazing performance while restoring the most accurate color details.
• Figure 4: Downstream task evaluation. We validate SAM SAM, DEQDet DEQDet, and LLaVA LLaVa on three downstream tasks. From top to bottom, we show the results of GT, hazy images, and our dehazed images on three downstream tasks, respectively. Green and red font colours indicate correct and incorrect descriptions. Best viewed in colour on the screen.