FALCON: Frequency Adjoint Link with CONtinuous Density Mask for Fast Single Image Dehazing

TL;DR

This work tackles the challenge of real-time single-image dehazing by balancing inference speed with high restoration quality. It introduces FALCON, which combines a Frequency Adjoint Link to expand the receptive field in the frequency domain with a Continuous Density Mask derived from the atmospheric scattering model to provide a haze-density prior and a differentiable loss. The approach achieves state-of-the-art PSNR/SSIM while delivering real-time FPS on standard GPUs across real-world and synthetic datasets, aided by a Density Map Loss that aligns haze density between hazy and dehazed images. By leveraging differentiable density pooling and a lightweight frequency-domain bottleneck, FALCON offers a practical, high-performance dehazing solution for applications like autonomous driving and surveillance in poor visibility conditions.

Abstract

Image dehazing, addressing atmospheric interference like fog and haze, remains a pervasive challenge crucial for robust vision applications such as surveillance and remote sensing under adverse visibility. While various methodologies have evolved from early works predicting transmission matrix and atmospheric light features to deep learning and dehazing networks, they innately prioritize dehazing quality metrics, neglecting the need for real-time applicability in time-sensitive domains like autonomous driving. This work introduces FALCON (Frequency Adjoint Link with CONtinuous density mask), a single-image dehazing system achieving state-of-the-art performance on both quality and speed. Particularly, we develop a novel bottleneck module, namely, Frequency Adjoint Link, operating in the frequency space to globally expand the receptive field with minimal growth in network size. Further, we leverage the underlying haze distribution based on the atmospheric scattering model via a Continuous Density Mask (CDM) which serves as a continuous-valued mask input prior and a differentiable auxiliary loss. Comprehensive experiments involving multiple state-of-the-art methods and ablation analysis demonstrate FALCON's exceptional performance in both dehazing quality and speed (i.e., >$180 frames-per-second), quantified by metrics such as FPS, PSNR, and SSIM.

Figures (10)

• Figure 1: (a) A Simplified Illustration of the FALCON Workflow. The purple arrow represents the operation of calculating the haze density map. We implement this process through an approach called Differentiable Density Pooling. (b) Analysis of dehazing performance (PSNR, Peak Signal-to-Noise Ratio) vs. dehazing speed (FPS, frames-per-second) on NH-Haze2 dataset with images of size 256$\times$256 using RTX 3090 GPU. For each method, its circle size is proportional to the FLOPS. The goal is to achieve both high PSNR for quality and FPS for speed. Our method FALCON achieves the highest PSNR (22.41 dB) with the fastest inference FPS (182.90 frames per second), enabling a real-time single image dehazing while achieving the best dehazing quality.
• Figure 2: Overall pipeline of our single image dehazing method, FALCON. (1) The input hazy image is concatenated with its haze density map, namely, Continuous Density Mask, as a haze prior. (2) Our main network takes the concatenated input and leverages our Frequency Adjoint Link (FAL) to efficiently exploit the frequency domain. (3) The output dehazed image is compared against the ground-truth image in the image space ($\mathcal{L}_{img}$), density map space ($\mathcal{L}_{map}$), and VGG-16 feature space ($\mathcal{L}_{per}$).
• Figure 3: Comparative visualization of dehazing results on Dense-Haze. The top row displays the overall results, while the bottom row shows a magnified view.
• Figure 4: Comparative visualization of dehazing results using the NH-Haze2 dataset. The top row displays the overall results, while the bottom row shows a magnified view.
• Figure 5: An ablation study showcasing the progressive enhancement of our method. From left to right: Input image, Base setting without our proposed enhancements, Base+FAL (Frequency Adjoint Link), Base+CDM (Continuous Density Mask), Base+Density Map Loss, our final method combining all enhancements, and Ground Truth (GT) image. Each column demonstrates the visual improvements achieved by incrementally integrating our proposed techniques.