Diffusion Models for Earth Observation Use-cases: from cloud removal to urban change detection

TL;DR

The paper investigates diffusion-based inpainting as a versatile tool for Earth Observation data. It presents three use cases—cloud removal, synthetic change-detection data generation, and urban replanning—to demonstrate the practical impact of diffusion models in EO. Two inpainting strategies, RePaint and input-concatenation, are demonstrated, with quantitative results showing SSIM 0.691 and PSNR 24.593 dB for cloud removal and a publicly released dataset for change detection. A text-conditioned StableDiffusion approach is shown for urban replanning, illustrating realistic EO content generation and potential downstream benefits.

Abstract

The advancements in the state of the art of generative Artificial Intelligence (AI) brought by diffusion models can be highly beneficial in novel contexts involving Earth observation data. After introducing this new family of generative models, this work proposes and analyses three use cases which demonstrate the potential of diffusion-based approaches for satellite image data. Namely, we tackle cloud removal and inpainting, dataset generation for change-detection tasks, and urban replanning.

Figures (5)

• Figure 1: Inpainting Variant 1: RePaint Lugmayr2022, where a partially known image $x^{\textrm{known}}$ is mixed with the generated sample $x_t$ using a mask $M$ to produce an inpainting at a time step $t-1$.
• Figure 2: Diagram of the concatenation inpainting approach for diffusion models.
• Figure 3: Selected samples computed on the cloud removal task. Note that these samples of real clouds come from a different set than our reported numerical results
• Figure 4: Example images from the generated change detection dataset.
• Figure 5: Example of urban replanning visualization, where a car park is replaced with a pedestrianised area with trees (top row) and another car park is replaced with a large swimming pool (bottom row).