Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Learning Techniques for Atmospheric Turbulence Removal: A Review

Paul Hill, Nantheera Anantrasirichai, Alin Achim, David Bull

TL;DR

This review provides a roadmap of how datasets and metrics together with currently used and newly developed deep learning methods could be used to develop the next generation of turbulence mitigation techniques.

Abstract

The influence of atmospheric turbulence on acquired imagery makes image interpretation and scene analysis extremely difficult and reduces the effectiveness of conventional approaches for classifying and tracking objects of interest in the scene. Restoring a scene distorted by atmospheric turbulence is also a challenging problem. The effect, which is caused by random, spatially varying perturbations, makes conventional model-based approaches difficult and, in most cases, impractical due to complexity and memory requirements. Deep learning approaches offer faster operation and are capable of implementation on small devices. This paper reviews the characteristics of atmospheric turbulence and its impact on acquired imagery. It compares the performance of various state-of-the-art deep neural networks, including Transformers, SWIN and Mamba, when used to mitigate spatio-temporal image distortions.

Deep Learning Techniques for Atmospheric Turbulence Removal: A Review

TL;DR

This review provides a roadmap of how datasets and metrics together with currently used and newly developed deep learning methods could be used to develop the next generation of turbulence mitigation techniques.

Abstract

The influence of atmospheric turbulence on acquired imagery makes image interpretation and scene analysis extremely difficult and reduces the effectiveness of conventional approaches for classifying and tracking objects of interest in the scene. Restoring a scene distorted by atmospheric turbulence is also a challenging problem. The effect, which is caused by random, spatially varying perturbations, makes conventional model-based approaches difficult and, in most cases, impractical due to complexity and memory requirements. Deep learning approaches offer faster operation and are capable of implementation on small devices. This paper reviews the characteristics of atmospheric turbulence and its impact on acquired imagery. It compares the performance of various state-of-the-art deep neural networks, including Transformers, SWIN and Mamba, when used to mitigate spatio-temporal image distortions.
Paper Structure (29 sections, 4 equations, 8 figures, 3 tables)

This paper contains 29 sections, 4 equations, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Atmospheric Turbulence Visual Effects
  3. Atmospheric Turbulence Effect Removal
  4. Paper Structure
  5. Characterising Atmospheric Turbulence
  6. Datasets
  7. Real datasets
  8. Real datasets with ground truth
  9. Real datasets without ground truth
  10. Synthetic Turbulence Generation
  11. 3D Wavefront Propagation Models using Ray Tracing
  12. Spatial Domain
  13. Frequency Domain
  14. Deep Learning Based Methods
  15. Evaluation of Simulated Turbulence Methods
  16. ...and 14 more sections

Figures (8)

  • Figure 1: Atmospheric distortions: Frame from the real atmospheric distortion "Van" sequence. The blue rectangle illustrates pixel displacements resulting in non-linear lines and the red rectangle illustrates the spatially varying blur effect.
  • Figure 2: Deformable mirror for adaptive optics based turbulence mitigation: adapted from [SteinbockMSc:2012].
  • Figure 3: Non-Deep Learning based Synthetic Turbulence Simulation Methods
  • Figure 4: Common workflow of atmospheric turbulence removal using multiple frames [Anantrasirichai:Atmospheric:2013].
  • Figure 5: Result comparison of "Car" sequence using UNet, EDVR and FFDNet [anantrasirichai2023atmospheric]
  • ...and 3 more figures