Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

STEREOFOG -- Computational DeFogging via Image-to-Image Translation on a real-world Dataset

Anton Pollak, Rajesh Menon

TL;DR

A binocular imaging system and Stereofog-an open-source dataset comprising 10,067 paired clear and foggy images, with a majority captured under dense fog conditions, emphasize the promise of machine-learning techniques in computational defogging across diverse fog conditions.

Abstract

Image-to-Image translation (I2I) is a subtype of Machine Learning (ML) that has tremendous potential in applications where two domains of images and the need for translation between the two exist, such as the removal of fog. For example, this could be useful for autonomous vehicles, which currently struggle with adverse weather conditions like fog. However, datasets for I2I tasks are not abundant and typically hard to acquire. Here, we introduce STEREOFOG, a dataset comprised of $10,067$ paired fogged and clear images, captured using a custom-built device, with the purpose of exploring I2I's potential in this domain. It is the only real-world dataset of this kind to the best of our knowledge. Furthermore, we apply and optimize the pix2pix I2I ML framework to this dataset. With the final model achieving an average Complex Wavelet-Structural Similarity (CW-SSIM) score of $0.76$, we prove the technique's suitability for the problem.

STEREOFOG -- Computational DeFogging via Image-to-Image Translation on a real-world Dataset

TL;DR

A binocular imaging system and Stereofog-an open-source dataset comprising 10,067 paired clear and foggy images, with a majority captured under dense fog conditions, emphasize the promise of machine-learning techniques in computational defogging across diverse fog conditions.

Abstract

Image-to-Image translation (I2I) is a subtype of Machine Learning (ML) that has tremendous potential in applications where two domains of images and the need for translation between the two exist, such as the removal of fog. For example, this could be useful for autonomous vehicles, which currently struggle with adverse weather conditions like fog. However, datasets for I2I tasks are not abundant and typically hard to acquire. Here, we introduce STEREOFOG, a dataset comprised of paired fogged and clear images, captured using a custom-built device, with the purpose of exploring I2I's potential in this domain. It is the only real-world dataset of this kind to the best of our knowledge. Furthermore, we apply and optimize the pix2pix I2I ML framework to this dataset. With the final model achieving an average Complex Wavelet-Structural Similarity (CW-SSIM) score of , we prove the technique's suitability for the problem.
Paper Structure (26 sections, 1 equation, 7 figures)

This paper contains 26 sections, 1 equation, 7 figures.

Table of Contents

  1. Introduction
  2. Methods
  3. State of the Art & Technology
  4. Fog datasets
  5. Haze datasets & Dehazing algorithms
  6. Image-to-image translation
  7. Image comparison metrics
  8. Binocular camera setup
  9. Dataset
  10. Machine Learning
  11. Dataset augmentation
  12. Model & Hyperparameters
  13. Evaluation
  14. Results
  15. Synthetic datasets
  16. ...and 11 more sections

Figures (7)

  • Figure 1: Overview of the Stereofog project. a): A diagram summarizing the work done in this work. b): Example results obtained by applying the pix2pix framework to the Stereofog dataset. Our approach works for a range of fog densities.
  • Figure 2: Binocular camera setup to capture foggy-clear image pairs. Top: Labelled CAD model. Bottom: Photographs of the setup.
  • Figure 3: Distribution of the Variances of the Laplacian ($\text{v}_\text{L}$) for the Stereofog dataset. Sample image pairs from the dataset with different fogginess levels and their respective $\text{v}_\text{L}$ values for context
  • Figure 4: Example results for the synthetic datasets. a): Cityscapes dataset from Uni. Tübingen Cordts2016Bernuth2019, b): Foggy Cityscapes dataset Cordts2016Sakaridis2018, c) Foggy CARLA dataset from Uni. Tübingen Dosovitskiy2017Bernuth2019
  • Figure 5: Results from the Stereofog dataset with the optimum set of hyperparameters.
  • ...and 2 more figures