Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Physics Informed and Data Driven Simulation of Underwater Images via Residual Learning

Tanmoy Mondal, Ricardo Mendoza, Lucas Drumetz

TL;DR

The paper addresses the challenge of underwater image degradation by proposing a physics-informed, data-driven simulator that uses a known dehazing-like image formation model together with learned residual factors to emulate unmodeled scattering and turbidity. It introduces a three-branch DenseNet-based network that simultaneously predicts depth, learns residual degradation, and directly predicts degraded images, enabling a differentiable, physically interpretable underwater image emulator. The authors construct a ground-truth-like dataset by simulating complex underwater formation equations and evaluate on NYU Depth v2 and Make3D, showing improvements over purely data-driven I2I methods and enabling inverse restoration through differentiable optimization. The work provides a framework for differentiable emulation of complex physical processes, with potential applications to other domains where partial physics is known and remaining effects are data-driven.

Abstract

In general, underwater images suffer from color distortion and low contrast, because light is attenuated and backscattered as it propagates through water (differently depending on wavelength and on the properties of the water body). An existing simple degradation model (similar to atmospheric image "hazing" effects), though helpful, is not sufficient to properly represent the underwater image degradation because there are unaccounted for and non-measurable factors e.g. scattering of light due to turbidity of water, reflective characteristics of turbid medium etc. We propose a deep learning-based architecture to automatically simulate the underwater effects where only a dehazing-like image formation equation is known to the network, and the additional degradation due to the other unknown factors if inferred in a data-driven way. We only use RGB images (because in real-time scenario depth image is not available) to estimate the depth image. For testing, we have proposed (due to the lack of real underwater image datasets) a complex image formation model/equation to manually generate images that resemble real underwater images (used as ground truth). However, only the classical image formation equation (the one used for image dehazing) is informed to the network. This mimics the fact that in a real scenario, the physics are never completely known and only simplified models are known. Thanks to the ground truth, generated by a complex image formation equation, we could successfully perform a qualitative and quantitative evaluation of proposed technique, compared to other purely data driven approaches

Physics Informed and Data Driven Simulation of Underwater Images via Residual Learning

TL;DR

The paper addresses the challenge of underwater image degradation by proposing a physics-informed, data-driven simulator that uses a known dehazing-like image formation model together with learned residual factors to emulate unmodeled scattering and turbidity. It introduces a three-branch DenseNet-based network that simultaneously predicts depth, learns residual degradation, and directly predicts degraded images, enabling a differentiable, physically interpretable underwater image emulator. The authors construct a ground-truth-like dataset by simulating complex underwater formation equations and evaluate on NYU Depth v2 and Make3D, showing improvements over purely data-driven I2I methods and enabling inverse restoration through differentiable optimization. The work provides a framework for differentiable emulation of complex physical processes, with potential applications to other domains where partial physics is known and remaining effects are data-driven.

Abstract

In general, underwater images suffer from color distortion and low contrast, because light is attenuated and backscattered as it propagates through water (differently depending on wavelength and on the properties of the water body). An existing simple degradation model (similar to atmospheric image "hazing" effects), though helpful, is not sufficient to properly represent the underwater image degradation because there are unaccounted for and non-measurable factors e.g. scattering of light due to turbidity of water, reflective characteristics of turbid medium etc. We propose a deep learning-based architecture to automatically simulate the underwater effects where only a dehazing-like image formation equation is known to the network, and the additional degradation due to the other unknown factors if inferred in a data-driven way. We only use RGB images (because in real-time scenario depth image is not available) to estimate the depth image. For testing, we have proposed (due to the lack of real underwater image datasets) a complex image formation model/equation to manually generate images that resemble real underwater images (used as ground truth). However, only the classical image formation equation (the one used for image dehazing) is informed to the network. This mimics the fact that in a real scenario, the physics are never completely known and only simplified models are known. Thanks to the ground truth, generated by a complex image formation equation, we could successfully perform a qualitative and quantitative evaluation of proposed technique, compared to other purely data driven approaches
Paper Structure (36 sections, 32 equations, 7 figures, 8 tables)

This paper contains 36 sections, 32 equations, 7 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image Formation Model
  4. Classical Simulation Model
  5. Underwater Image Simulation Model
  6. Proposed Method
  7. Network Architecture
  8. Learning Loss Function
  9. Technique 1:
  10. Variant 1 :
  11. Variant 2 :
  12. Technique 2:
  13. Technique 3:
  14. Experimental Results
  15. Datasets
  16. ...and 21 more sections

Figures (7)

  • Figure 1: Examples of degradations that can be simulated using our approach on NYU Depth v2 dataset. Left: First row - underwater degradation (with increased water attenuation); Second row - diverse attenuation-lighting-scattering configurations. Right: first row - scatter-enriched fog, second row - smoke-like degraded environments.
  • Figure 1: (a) The attenuation coefficients ($\beta$) of Jerlov water types. The solid lines corresponds to open ocean water types while the dashed lines mark coastal water types. (b) For the case of different water types, the simulation of the appearance of white surface, viewed at depth of $1-20$m. (figures are taken from Berman2017)
  • Figure 2: The proposed network architecture
  • Figure 2: (a) The training loss curve of Technique-2 and it's variants (b) The training loss curve of Technique-3 and it's variants
  • Figure 3: Qualitative Measures obtained by Proposed Technique 1 : (a) Original RGB images (i.e. $J_c(\mathbf{x})$ in Equation \ref{['eq_1']}) (b) Ground Truth of "Initial Degraded Image" (i.e. $I_c(\mathbf{x})$ in Equation \ref{['eq_1']}) (c) Ground Truth of "Simulated Underwater Image" (i.e. $I^{sct}_c(\textbf{x})$ in Equation \ref{['eq_3_3']}) (d) Predicted "Initial Degraded Image" (i.e. $I^{Degraded}_{Initial}$ in Fig. \ref{['fig:archi_model']}) (e) Predicted "Residual Image" (i.e. $I^{Residue}$ in Fig. \ref{['fig:archi_model']}) (f) Predicted "Simulated Underwater Image" (i.e. $\hat{I}^{Simulated}_{Predicted}$ in Fig. \ref{['fig:archi_model']}).
  • ...and 2 more figures