Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cross-sensor super-resolution of irregularly sampled Sentinel-2 time series

Aimi Okabayashi, Nicolas Audebert, Simon Donike, Charlotte Pelletier

TL;DR

The paper tackles reconstructing high-resolution Sentinel-2 time-series imagery by cross-sensor MISR using SPOT-6 as the HR reference. It extends diffusion-based SR (SRDiff) with MISR backbones and introduces a time-equivariant fusion module (L-TAE) to handle irregular time sampling, enabling HR outputs at any target date within the LR sequence. A new BreizhSR dataset (Breizh region) of 52,255 S2–SPOT-6 pairs enables real-world evaluation at 4× resolution. Results show MISR with time-aware fusion improves both pixel- and perceptual-based metrics, though a trade-off between spectral fidelity and perceptual quality emerges, suggesting future work on joint losses and full-spectrum EO SR.

Abstract

Satellite imaging generally presents a trade-off between the frequency of acquisitions and the spatial resolution of the images. Super-resolution is often advanced as a way to get the best of both worlds. In this work, we investigate multi-image super-resolution of satellite image time series, i.e. how multiple images of the same area acquired at different dates can help reconstruct a higher resolution observation. In particular, we extend state-of-the-art deep single and multi-image super-resolution algorithms, such as SRDiff and HighRes-net, to deal with irregularly sampled Sentinel-2 time series. We introduce BreizhSR, a new dataset for 4x super-resolution of Sentinel-2 time series using very high-resolution SPOT-6 imagery of Brittany, a French region. We show that using multiple images significantly improves super-resolution performance, and that a well-designed temporal positional encoding allows us to perform super-resolution for different times of the series. In addition, we observe a trade-off between spectral fidelity and perceptual quality of the reconstructed HR images, questioning future directions for super-resolution of Earth Observation data.

Cross-sensor super-resolution of irregularly sampled Sentinel-2 time series

TL;DR

The paper tackles reconstructing high-resolution Sentinel-2 time-series imagery by cross-sensor MISR using SPOT-6 as the HR reference. It extends diffusion-based SR (SRDiff) with MISR backbones and introduces a time-equivariant fusion module (L-TAE) to handle irregular time sampling, enabling HR outputs at any target date within the LR sequence. A new BreizhSR dataset (Breizh region) of 52,255 S2–SPOT-6 pairs enables real-world evaluation at 4× resolution. Results show MISR with time-aware fusion improves both pixel- and perceptual-based metrics, though a trade-off between spectral fidelity and perceptual quality emerges, suggesting future work on joint losses and full-spectrum EO SR.

Abstract

Satellite imaging generally presents a trade-off between the frequency of acquisitions and the spatial resolution of the images. Super-resolution is often advanced as a way to get the best of both worlds. In this work, we investigate multi-image super-resolution of satellite image time series, i.e. how multiple images of the same area acquired at different dates can help reconstruct a higher resolution observation. In particular, we extend state-of-the-art deep single and multi-image super-resolution algorithms, such as SRDiff and HighRes-net, to deal with irregularly sampled Sentinel-2 time series. We introduce BreizhSR, a new dataset for 4x super-resolution of Sentinel-2 time series using very high-resolution SPOT-6 imagery of Brittany, a French region. We show that using multiple images significantly improves super-resolution performance, and that a well-designed temporal positional encoding allows us to perform super-resolution for different times of the series. In addition, we observe a trade-off between spectral fidelity and perceptual quality of the reconstructed HR images, questioning future directions for super-resolution of Earth Observation data.
Paper Structure (15 sections, 4 equations, 6 figures, 3 tables)

This paper contains 15 sections, 4 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed methods
  4. From single- to multi-image super resolution
  5. Time-Equivariant Super-Resolution
  6. Experimental setup
  7. The BreizhSR dataset
  8. Pre-processing
  9. Implementation and architectures
  10. Evaluation metrics
  11. Results and discussion
  12. Quantitative results
  13. Cloud impact
  14. Conclusion
  15. Acknowledgements

Figures (6)

  • Figure 1: Multi-image super-resolution with an upscaling factor of 4. The irregular low-resolution (LR) satellite time series is used to predict a super-resolved (SR) image for a given acquisition date.
  • Figure 2: Study area with the minimal time difference of Sentinel-2 and SPOT-6 acquisitions.
  • Figure 3: Boxplots of MAE results as a function of the time difference between the acquisitions of the SPOT-6 image and the closest Sentinel-2 image. From left to right: time difference of less than 10 days (10 758 images), between 10 and 30 days (3 633 images), over 30 days (630 images).
  • Figure 4: Multi-image super-resolution at different time steps using HighRes-net L-TAE. SR images follow changes in the LR series.
  • Figure 5: Visual results with the different models. SISR: RRDB, SRDiff (RRDB), MISR: HighRes-net recursive fusion, HighRes-net L-TAE, RRDB L-TAE, SRDiff HighRes-net L-TAE. HR acquisition date: 2018-06-22.
  • ...and 1 more figures