Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-Quality Cloud-Free Optical Image Synthesis Using Multi-Temporal SAR and Contaminated Optical Data

Chenxi Duan

TL;DR

This work tackles the problem of missing optical data due to cloud cover and long revisit cycles by introducing CRSynthNet, a GAN-based framework that fuses multi-temporal SAR data with cloud-contaminated optical inputs. It features a DownUpBlock and FusionAttention in the generator, a multi-scale Swin-Transformer-based encoder/decoder, and a three-scale discriminator, all optimized with a loss combining $L_{ls}$ and $L_{sim}$ (including $L_{VGG}$, $L_{CS}$, and $L_{MS-SSIM}$). The authors also introduce the TCSEN12 dataset from the Zhengzhou flood region to evaluate cloud-resilient synthesis in a real-world, change-rich scenario. Empirical results show CRSynthNet achieving $PSNR=26.978$, $SSIM=0.648$, $RMSE=0.050$, $MAE=0.041$, and $FID=72.789$, outperforming several baselines and demonstrating practical utility for cloud-impacted optical image synthesis.

Abstract

Addressing gaps caused by cloud cover and the long revisit cycle of satellites is vital for providing essential data to support remote sensing applications. This paper tackles the challenges of missing optical data synthesis, particularly in complex scenarios with cloud cover. We propose CRSynthNet, a novel image synthesis network that incorporates innovative designed modules such as the DownUp Block and Fusion Attention to enhance accuracy. Experimental results validate the effectiveness of CRSynthNet, demonstrating substantial improvements in restoring structural details, preserving spectral consist, and achieving superior visual effects that far exceed those produced by comparison methods. It achieves quantitative improvements across multiple metrics: a peak signal-to-noise ratio (PSNR) of 26.978, a structural similarity index measure (SSIM) of 0.648, and a root mean square error (RMSE) of 0.050. Furthermore, this study creates the TCSEN12 dataset, a valuable resource specifically designed to address cloud cover challenges in missing optical data synthesis study. The dataset uniquely includes cloud-covered images and leverages earlier image to predict later image, offering a realistic representation of real-world scenarios. This study offer practical method and valuable resources for optical satellite image synthesis task.

High-Quality Cloud-Free Optical Image Synthesis Using Multi-Temporal SAR and Contaminated Optical Data

TL;DR

This work tackles the problem of missing optical data due to cloud cover and long revisit cycles by introducing CRSynthNet, a GAN-based framework that fuses multi-temporal SAR data with cloud-contaminated optical inputs. It features a DownUpBlock and FusionAttention in the generator, a multi-scale Swin-Transformer-based encoder/decoder, and a three-scale discriminator, all optimized with a loss combining and (including , , and ). The authors also introduce the TCSEN12 dataset from the Zhengzhou flood region to evaluate cloud-resilient synthesis in a real-world, change-rich scenario. Empirical results show CRSynthNet achieving , , , , and , outperforming several baselines and demonstrating practical utility for cloud-impacted optical image synthesis.

Abstract

Addressing gaps caused by cloud cover and the long revisit cycle of satellites is vital for providing essential data to support remote sensing applications. This paper tackles the challenges of missing optical data synthesis, particularly in complex scenarios with cloud cover. We propose CRSynthNet, a novel image synthesis network that incorporates innovative designed modules such as the DownUp Block and Fusion Attention to enhance accuracy. Experimental results validate the effectiveness of CRSynthNet, demonstrating substantial improvements in restoring structural details, preserving spectral consist, and achieving superior visual effects that far exceed those produced by comparison methods. It achieves quantitative improvements across multiple metrics: a peak signal-to-noise ratio (PSNR) of 26.978, a structural similarity index measure (SSIM) of 0.648, and a root mean square error (RMSE) of 0.050. Furthermore, this study creates the TCSEN12 dataset, a valuable resource specifically designed to address cloud cover challenges in missing optical data synthesis study. The dataset uniquely includes cloud-covered images and leverages earlier image to predict later image, offering a realistic representation of real-world scenarios. This study offer practical method and valuable resources for optical satellite image synthesis task.

Paper Structure

This paper contains 16 sections, 4 equations, 10 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Dataset and Method
  3. TCSEN12 Dataset
  4. Method
  5. Generator
  6. Discriminator
  7. Loss function
  8. Experiments
  9. Implementation Details
  10. Quantitative Evaluation
  11. Qualitative Evaluation
  12. Ablation Experiments
  13. Discussion
  14. Efficiency of CRSynthNet
  15. Limitations
  16. ...and 1 more sections

Figures (10)

  • Figure 1: The architecture of the proposed generator. The detailed structure of each block is in Figure \ref{['method_downup']} to Figure \ref{['method_decorder']}.
  • Figure 2: Detailed components of the DownUp block. The ConvMishBlock is illustrated in Figure \ref{['method_Mish']}.
  • Figure 3: The detailed design of three convolutional blocks.
  • Figure 5: The detailed architecture of Decoder.
  • Figure 6: The architecture of Discriminator with the structure of D1.
  • ...and 5 more figures