Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prompt-based test-time real image dehazing: a novel pipeline

Zixuan Chen, Zewei He, Ziqian Lu, Xuecheng Sun, Zhe-Ming Lu

TL;DR

A novel pipeline called Prompt-based Test-Time Dehazing (PTTD) is presented to help generate visually pleasing results of real-captured hazy images during the inference phase, achieving superior performance against state-of-the-art dehazing methods in real-world scenarios.

Abstract

Existing methods attempt to improve models' generalization ability on real-world hazy images by exploring well-designed training schemes (\eg, CycleGAN, prior loss). However, most of them need very complicated training procedures to achieve satisfactory results. For the first time, we present a novel pipeline called Prompt-based Test-Time Dehazing (PTTD) to help generate visually pleasing results of real-captured hazy images during the inference phase. We experimentally observe that given a dehazing model trained on synthetic data, fine-tuning the statistics (\ie, mean and standard deviation) of encoding features is able to narrow the domain gap, boosting the performance of real image dehazing. Accordingly, we first apply a prompt generation module (PGM) to generate a visual prompt, which is the reference of appropriate statistical perturbations for mean and standard deviation. Then, we employ a feature adaptation module (FAM) into the existing dehazing models for adjusting the original statistics with the guidance of the generated prompt. PTTD is model-agnostic and can be equipped with various state-of-the-art dehazing models trained on synthetic hazy-clean pairs to tackle the real image dehazing task. Extensive experimental results demonstrate that our PTTD is effective, achieving superior performance against state-of-the-art dehazing methods in real-world scenarios. The code is available at \url{https://github.com/cecret3350/PTTD-Dehazing}.

Prompt-based test-time real image dehazing: a novel pipeline

TL;DR

A novel pipeline called Prompt-based Test-Time Dehazing (PTTD) is presented to help generate visually pleasing results of real-captured hazy images during the inference phase, achieving superior performance against state-of-the-art dehazing methods in real-world scenarios.

Abstract

Existing methods attempt to improve models' generalization ability on real-world hazy images by exploring well-designed training schemes (\eg, CycleGAN, prior loss). However, most of them need very complicated training procedures to achieve satisfactory results. For the first time, we present a novel pipeline called Prompt-based Test-Time Dehazing (PTTD) to help generate visually pleasing results of real-captured hazy images during the inference phase. We experimentally observe that given a dehazing model trained on synthetic data, fine-tuning the statistics (\ie, mean and standard deviation) of encoding features is able to narrow the domain gap, boosting the performance of real image dehazing. Accordingly, we first apply a prompt generation module (PGM) to generate a visual prompt, which is the reference of appropriate statistical perturbations for mean and standard deviation. Then, we employ a feature adaptation module (FAM) into the existing dehazing models for adjusting the original statistics with the guidance of the generated prompt. PTTD is model-agnostic and can be equipped with various state-of-the-art dehazing models trained on synthetic hazy-clean pairs to tackle the real image dehazing task. Extensive experimental results demonstrate that our PTTD is effective, achieving superior performance against state-of-the-art dehazing methods in real-world scenarios. The code is available at \url{https://github.com/cecret3350/PTTD-Dehazing}.
Paper Structure (16 sections, 3 equations, 10 figures, 4 tables, 1 algorithm)

This paper contains 16 sections, 3 equations, 10 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Single Image Dehazing
  4. Real Image Dehazing
  5. Methodology
  6. Observation and Motivation
  7. Pipeline Overview
  8. Prompt Generation Module
  9. Feature Adaptation Module
  10. Experiments
  11. Experimental Configuration
  12. Experiments on Real-captured Datasets with Labels
  13. Experiments on RTTS and Fattal's
  14. Ablation Study
  15. Inference Time
  16. ...and 1 more sections

Figures (10)

  • Figure 1: Different frameworks and their results on real-world hazy images. (a) Directly apply model trained on synthetic data to real hazy images (e.g., AECRNet wu2021CVPR); (b) Use synthetic and real data together to train the model and then apply it to real hazy images (e.g., PSD Chen2021CVPR, DAD Shao2020CVPR); (c) Our proposed PTTD; (d) A real hazy image; and (e-h) Processing results of state-of-the-art (SOTA) methods (AECRNet wu2021CVPR, PSD Chen2021CVPR, DAD Shao2020CVPR) and our proposed PTTD (by adopting pre-trained AECRNet wu2021CVPR). It can be observed that AECRNet-PTTD achieves very promising results.
  • Figure 2: (a) is a real hazy input; (b) is the processing result of AECRNet wu2021CVPR, which is trained on synthetic data; (c) and (d) are the results by fine-tuning the mean $\mu$; (e) and (f) are the results by fine-tuning the standard deviation $\sigma$. In this experiment, the perturbation $\Delta$ is set to a small constant, i.e., $\Delta = 0.005$.
  • Figure 3: (a) The overall architecture of our Prompt-based Test-Time Dehazing (PTTD); (b) The proposed prompt generation module (PGM).
  • Figure 4: (a) A real hazy image $x_r$; (b) A haze-free image $y_s$; (c) Prompt via direct ILN (adaIN); (d) Prompt via ILN with partition; (e) Prompt via CBILN with partition.
  • Figure 5: Dehazing results of various methods on O-HAZE. We choose AECRNet-PTTD to compare with SOTA dehazing methods. Please zoom in on screen for a better view.
  • ...and 5 more figures