Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Atmospheric Turbulence Removal with Video Sequence Deep Visual Priors

P. Hill, N. Anantrasirichai, A. Achim, D. R. Bull

TL;DR

Atmospheric turbulence distorts long-range video, challenging interpretation and automated analytics. The authors present a self-supervised pipeline that extends the Deep Image Prior with temporal information (pixel shuffling and a sliding window) to learn spatio-temporal priors from the data sequence alone. They apply acceleration (DRP), partial weight freezing, latent variable prediction, and early stopping to build an efficient turbulence mitigation system that works on raw or pre-processed inputs. Quantitative results using no-reference BIQI and a background-variance metric show consistent quality gains, and qualitative analysis reveals reduced artefacts compared with state-of-the-art model-based CLEAR, especially when combined. The approach generalizes to arbitrary turbulence sequences without external training data, offering a practical tool for surveillance and recovery of distorted video.

Abstract

Atmospheric turbulence poses a challenge for the interpretation and visual perception of visual imagery due to its distortion effects. Model-based approaches have been used to address this, but such methods often suffer from artefacts associated with moving content. Conversely, deep learning based methods are dependent on large and diverse datasets that may not effectively represent any specific content. In this paper, we address these problems with a self-supervised learning method that does not require ground truth. The proposed method is not dependent on any dataset outside of the single data sequence being processed but is also able to improve the quality of any input raw sequences or pre-processed sequences. Specifically, our method is based on an accelerated Deep Image Prior (DIP), but integrates temporal information using pixel shuffling and a temporal sliding window. This efficiently learns spatio-temporal priors leading to a system that effectively mitigates atmospheric turbulence distortions. The experiments show that our method improves visual quality results qualitatively and quantitatively.

Atmospheric Turbulence Removal with Video Sequence Deep Visual Priors

TL;DR

Atmospheric turbulence distorts long-range video, challenging interpretation and automated analytics. The authors present a self-supervised pipeline that extends the Deep Image Prior with temporal information (pixel shuffling and a sliding window) to learn spatio-temporal priors from the data sequence alone. They apply acceleration (DRP), partial weight freezing, latent variable prediction, and early stopping to build an efficient turbulence mitigation system that works on raw or pre-processed inputs. Quantitative results using no-reference BIQI and a background-variance metric show consistent quality gains, and qualitative analysis reveals reduced artefacts compared with state-of-the-art model-based CLEAR, especially when combined. The approach generalizes to arbitrary turbulence sequences without external training data, offering a practical tool for surveillance and recovery of distorted video.

Abstract

Atmospheric turbulence poses a challenge for the interpretation and visual perception of visual imagery due to its distortion effects. Model-based approaches have been used to address this, but such methods often suffer from artefacts associated with moving content. Conversely, deep learning based methods are dependent on large and diverse datasets that may not effectively represent any specific content. In this paper, we address these problems with a self-supervised learning method that does not require ground truth. The proposed method is not dependent on any dataset outside of the single data sequence being processed but is also able to improve the quality of any input raw sequences or pre-processed sequences. Specifically, our method is based on an accelerated Deep Image Prior (DIP), but integrates temporal information using pixel shuffling and a temporal sliding window. This efficiently learns spatio-temporal priors leading to a system that effectively mitigates atmospheric turbulence distortions. The experiments show that our method improves visual quality results qualitatively and quantitatively.
Paper Structure (10 sections, 5 equations, 3 figures)

This paper contains 10 sections, 5 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Proposed methodology
  3. Deep Image Prior (DIP)
  4. Overfitting and Early Stopping (ES)
  5. DIP Acceleration
  6. Architecture and implementation
  7. Experiments and discussion
  8. Metrics
  9. Results and discussion
  10. Conclusion

Figures (3)

  • Figure 1: Latent Variable / Batch Normalisation weights prediction {$\theta_{BN}, \boldsymbol{z}$}
  • Figure 2: Architecture of our developed image sequence atmospheric mitigation method
  • Figure : Table 1. Results: BIQI and Variance results for input together with results for CLEAR, our method and combined our and CLEAR methods (first 5 frames of each sequence). BIQI results show the mean and standard deviations for each set of frames. The higher the BIQI value the lower the quality.