Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Not every day is a sunny day: Synthetic cloud injection for deep land cover segmentation robustness evaluation across data sources

Sara Mobsite, Renaud Hostache, Laure Berti Equille, Emmanuel Roux, Joris Guerin

TL;DR

The paper tackles the gap between ideal cloud-free satellite datasets and real-world cloud-prone conditions by introducing a synthetic cloud injection framework to evaluate land cover segmentation across SAR and optical data sources. It also proposes a decoder-level injection of Normalized Difference Indices to recover spatial details lost during encoder downsampling, improving performance for architectures like U-Net and DeepLabV3. Experimental results show modest gains from NDIs on cloud-free data and substantial improvements when models are trained with cloud-injected data, especially for optical-only and SAR–optical configurations. The findings underscore the value of cloud-aware training and radar–optical fusion in robust land cover mapping under atmospheric disturbances, with practical implications for tropical region monitoring and multi-sensor data fusion strategies.

Abstract

Supervised deep learning for land cover semantic segmentation (LCS) relies on labeled satellite data. However, most existing Sentinel-2 datasets are cloud-free, which limits their usefulness in tropical regions where clouds are common. To properly evaluate the extent of this problem, we developed a cloud injection algorithm that simulates realistic cloud cover, allowing us to test how Sentinel-1 radar data can fill in the gaps caused by cloud-obstructed optical imagery. We also tackle the issue of losing spatial and/or spectral details during encoder downsampling in deep networks. To mitigate this loss, we propose a lightweight method that injects Normalized Difference Indices (NDIs) into the final decoding layers, enabling the model to retain key spatial features with minimal additional computation. Injecting NDIs enhanced land cover segmentation performance on the DFC2020 dataset, yielding improvements of 1.99% for U-Net and 2.78% for DeepLabV3 on cloud-free imagery. Under cloud-covered conditions, incorporating Sentinel-1 data led to significant performance gains across all models compared to using optical data alone, highlighting the effectiveness of radar-optical fusion in challenging atmospheric scenarios.

Not every day is a sunny day: Synthetic cloud injection for deep land cover segmentation robustness evaluation across data sources

TL;DR

The paper tackles the gap between ideal cloud-free satellite datasets and real-world cloud-prone conditions by introducing a synthetic cloud injection framework to evaluate land cover segmentation across SAR and optical data sources. It also proposes a decoder-level injection of Normalized Difference Indices to recover spatial details lost during encoder downsampling, improving performance for architectures like U-Net and DeepLabV3. Experimental results show modest gains from NDIs on cloud-free data and substantial improvements when models are trained with cloud-injected data, especially for optical-only and SAR–optical configurations. The findings underscore the value of cloud-aware training and radar–optical fusion in robust land cover mapping under atmospheric disturbances, with practical implications for tropical region monitoring and multi-sensor data fusion strategies.

Abstract

Supervised deep learning for land cover semantic segmentation (LCS) relies on labeled satellite data. However, most existing Sentinel-2 datasets are cloud-free, which limits their usefulness in tropical regions where clouds are common. To properly evaluate the extent of this problem, we developed a cloud injection algorithm that simulates realistic cloud cover, allowing us to test how Sentinel-1 radar data can fill in the gaps caused by cloud-obstructed optical imagery. We also tackle the issue of losing spatial and/or spectral details during encoder downsampling in deep networks. To mitigate this loss, we propose a lightweight method that injects Normalized Difference Indices (NDIs) into the final decoding layers, enabling the model to retain key spatial features with minimal additional computation. Injecting NDIs enhanced land cover segmentation performance on the DFC2020 dataset, yielding improvements of 1.99% for U-Net and 2.78% for DeepLabV3 on cloud-free imagery. Under cloud-covered conditions, incorporating Sentinel-1 data led to significant performance gains across all models compared to using optical data alone, highlighting the effectiveness of radar-optical fusion in challenging atmospheric scenarios.

Paper Structure

This paper contains 12 sections, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. SAR and Optical Data Fusion Architecture
  5. Improving Segmentation with NDIs
  6. Synthetic Cloud Generation and Injection into Optical Data
  7. Experiments
  8. Dataset
  9. Implementation Details
  10. Results and Discussion
  11. The Impact of NDIs Decoder-Level Injection on Model Performance
  12. The Impact of Cloud Injection on Model Performance

Figures (2)

  • Figure 1: NDIs injection in the decoder of a pretrained autoencoder
  • Figure 2: Example of synthetic cloud generation based on a binary cloud location mask and cloud optical data