Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comprehensive Survey on Synthetic Infrared Image synthesis

Avinash Upadhyay, Manoj sharma, Prerana Mukherjee, Amit Singhal, Brejesh Lall

TL;DR

This survey tackles the scarcity of real infrared data by surveying both physics-based and learning-based synthetic IR image and video generation methods across the 0.7–14 µm range. It synthesizes atmospheric transfer modeling tools (e.g., DISORT, MODTRAN) and radiometric pipelines, catalogs extensive IR datasets, and catalogs synthetic scene tools and DL-based translation approaches (e.g., CycleGAN, Pix2Pix, SIR-GAN, V2IR-GAN). The work highlights challenges in realism, validation, and domain gap, and outlines future avenues such as richer simulators, larger multimodal datasets, and physics-informed learning for IR scene understanding with potential impact on defense, surveillance, and autonomous systems.

Abstract

Synthetic infrared (IR) scene and target generation is an important computer vision problem as it allows the generation of realistic IR images and targets for training and testing of various applications, such as remote sensing, surveillance, and target recognition. It also helps reduce the cost and risk associated with collecting real-world IR data. This survey paper aims to provide a comprehensive overview of the conventional mathematical modelling-based methods and deep learning-based methods used for generating synthetic IR scenes and targets. The paper discusses the importance of synthetic IR scene and target generation and briefly covers the mathematics of blackbody and grey body radiations, as well as IR image-capturing methods. The potential use cases of synthetic IR scenes and target generation are also described, highlighting the significance of these techniques in various fields. Additionally, the paper explores possible new ways of developing new techniques to enhance the efficiency and effectiveness of synthetic IR scenes and target generation while highlighting the need for further research to advance this field.

A Comprehensive Survey on Synthetic Infrared Image synthesis

TL;DR

This survey tackles the scarcity of real infrared data by surveying both physics-based and learning-based synthetic IR image and video generation methods across the 0.7–14 µm range. It synthesizes atmospheric transfer modeling tools (e.g., DISORT, MODTRAN) and radiometric pipelines, catalogs extensive IR datasets, and catalogs synthetic scene tools and DL-based translation approaches (e.g., CycleGAN, Pix2Pix, SIR-GAN, V2IR-GAN). The work highlights challenges in realism, validation, and domain gap, and outlines future avenues such as richer simulators, larger multimodal datasets, and physics-informed learning for IR scene understanding with potential impact on defense, surveillance, and autonomous systems.

Abstract

Synthetic infrared (IR) scene and target generation is an important computer vision problem as it allows the generation of realistic IR images and targets for training and testing of various applications, such as remote sensing, surveillance, and target recognition. It also helps reduce the cost and risk associated with collecting real-world IR data. This survey paper aims to provide a comprehensive overview of the conventional mathematical modelling-based methods and deep learning-based methods used for generating synthetic IR scenes and targets. The paper discusses the importance of synthetic IR scene and target generation and briefly covers the mathematics of blackbody and grey body radiations, as well as IR image-capturing methods. The potential use cases of synthetic IR scenes and target generation are also described, highlighting the significance of these techniques in various fields. Additionally, the paper explores possible new ways of developing new techniques to enhance the efficiency and effectiveness of synthetic IR scenes and target generation while highlighting the need for further research to advance this field.
Paper Structure (41 sections, 7 equations, 13 figures, 5 tables)

This paper contains 41 sections, 7 equations, 13 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Basics of Infrared Imagery
  3. Types of IR
  4. Infrared Radiometry and Detection
  5. Heat Equilibrium equation and surface temperature calculations
  6. Spectral radiance of body
  7. Radiance to Detector Voltage
  8. Tools and Algorithms for modelling atmospheric transfer function
  9. DISORT (Discrete Ordinate Radiative Transfer)
  10. RTTOV (Radiative Transfer for TOVS)
  11. ARTS (Atmospheric Radiative Transfer Simulator)
  12. TES (Transmittance Estimation from the Surface)
  13. LOWTRAN and MODTRAN
  14. ATRANATRAN
  15. Datasets
  16. ...and 26 more sections

Figures (13)

  • Figure 1: Pictorial representation of an Infrared System.
  • Figure 2: Atmospheric transmission pattern generated by MODTRAN module for the wavelength range of MWIR 3$\mu m$ to 5$\mu m$. Atmosphere Model used is US Standard 1976, ground temperature is 294.2K, Aerosol model used is Urban with a visibility of 23km, sensor altitute is 5km and zenith is 90 degrees.
  • Figure 3: Image snippets from different datasets.
  • Figure 4: IR image/video synthesis methods classification.
  • Figure 5: Conversion of one IR band to another IR band by first estimating the temperature of objects in the image, then compensating the temperature and predicting IR signature in different bands using method proposed by Bae et al Bae2019. Image courtesy Bae2019
  • ...and 8 more figures