Cross-Sensor Adversarial Domain Adaptation of Landsat-8 and Proba-V images for Cloud Detection

TL;DR

The paper tackles cross-sensor data shifts between Landsat-8 and Proba-V to improve cloud detection by applying an unpaired, pixel-level domain adaptation based on CyCADA. It introduces an upscaling step (L8 to LU) and a subsequent adversarial adaptation (PV to LU) with cycle, identity, and segmentation-consistency losses, enabling transfer of Landsat-8 cloud detectors to Proba-V data. Results show the adaptation reduces radiometric and textural differences while preserving spatial-spectral integrity, and markedly enhances Proba-V cloud detection when trained on Landsat-8 data. The approach is sensor-agnostic and extendable to other RS sensor pairs and tasks.

Abstract

The number of Earth observation satellites carrying optical sensors with similar characteristics is constantly growing. Despite their similarities and the potential synergies among them, derived satellite products are often developed for each sensor independently. Differences in retrieved radiances lead to significant drops in accuracy, which hampers knowledge and information sharing across sensors. This is particularly harmful for machine learning algorithms, since gathering new ground truth data to train models for each sensor is costly and requires experienced manpower. In this work, we propose a domain adaptation transformation to reduce the statistical differences between images of two satellite sensors in order to boost the performance of transfer learning models. The proposed methodology is based on the Cycle Consistent Generative Adversarial Domain Adaptation (CyCADA) framework that trains the transformation model in an unpaired manner. In particular, Landsat-8 and Proba-V satellites, which present different but compatible spatio-spectral characteristics, are used to illustrate the method. The obtained transformation significantly reduces differences between the image datasets while preserving the spatial and spectral information of adapted images, which is hence useful for any general purpose cross-sensor application. In addition, the training of the proposed adversarial domain adaptation model can be modified to improve the performance in a specific remote sensing application, such as cloud detection, by including a dedicated term in the cost function. Results show that, when the proposed transformation is applied, cloud detection models trained in Landsat-8 data increase cloud detection accuracy in Proba-V.

Figures (15)

• Figure 1: Close-in-time acquisitions of Landsat-8 and Proba-V satellites. Landsat-8 image is transformed and upscaled to resemble the optical characteristics of the Proba-V sensor; however, differences in radiometry and texture between images still remain. First row: Missouri river in North America (2016-04-20). Second and third row: North West Pacific coast, North America (2016-05-20 and 2016-04-20).
• Figure 2: Transfer learning and adaptation scheme: Landsat-8 and Proba-V datasets and how they are transformed between the three different domains. The transformations look for adaptation between the domains: $U$ is the upscaling transformation applied to Landsat-8 to resemble the Proba-V instrument characteristics (sec. \ref{['sec:pbda']}); and $A$ adapts from the Proba-V domain to the Landsat-8 upscaled domain (sec. \ref{['sec:gada']}).
• Figure 3: Upscaling transformation ($U$ in Fig. \ref{['fig:methodology']}) applied to Landsat-8 in order to resemble the Proba-V instrument characteristics.
• Figure 4: Spectral response of Landsat-8 and Proba-V channels.
• Figure 5: Scheme of the forward passes for the training procedure of the proposed cycle consistent adversarial domain adaptation method. The four networks ($G_{\mathrm{PV}\to\mathrm{LU}}$, $G_{\mathrm{LU}\to\mathrm{PV}}$, $D_{\mathrm{PV}}$, $D_{\mathrm{LU}}$) have a different color. Losses are depicted with circles and their fill color corresponds to the color of the network that they penalize.