Large Scale Masked Autoencoding for Reducing Label Requirements on SAR Data

TL;DR

This work applies a self-supervised pretraining scheme, masked autoencoding, to SAR amplitude data, and tunes the pretrained weights on two downstream tasks crucial to monitoring climate change - vegetation cover prediction and land cover classification, allowing local communities and organizations to deploy tailored solutions for rapid, accurate monitoring of climate change effects.

Abstract

Satellite-based remote sensing is instrumental in the monitoring and mitigation of the effects of anthropogenic climate change. Large scale, high resolution data derived from these sensors can be used to inform intervention and policy decision making, but the timeliness and accuracy of these interventions is limited by use of optical data, which cannot operate at night and is affected by adverse weather conditions. Synthetic Aperture Radar (SAR) offers a robust alternative to optical data, but its associated complexities limit the scope of labelled data generation for traditional deep learning. In this work, we apply a self-supervised pretraining scheme, masked autoencoding, to SAR amplitude data covering 8.7\% of the Earth's land surface area, and tune the pretrained weights on two downstream tasks crucial to monitoring climate change - vegetation cover prediction and land cover classification. We show that the use of this pretraining scheme reduces labelling requirements for the downstream tasks by more than an order of magnitude, and that this pretraining generalises geographically, with the performance gain increasing when tuned downstream on regions outside the pretraining set. Our findings significantly advance climate change mitigation by facilitating the development of task and region-specific SAR models, allowing local communities and organizations to deploy tailored solutions for rapid, accurate monitoring of climate change effects.

Figures (6)

• Figure 1: Qualitative results for ESAWC land cover classification: Land cover classification for ESAWC on data from Europe (top row) and South America (bottom row).
• Figure 2: Correlation plots for MODISVEG prediction finetuning the pretrained model with 100% of the labelled data: Europe (left column) and South America (right column). Linear fits obtained by ordinary least squares (OLS).
• Figure A.1: Data split: Geographic bands for training/validation/test sets at a ratio of 60:20:20.
• Figure A.2: Masked SAR Reconstruction: Masked autoencoder-based reconstruction of SAR amplitude imagery from the validation set. Within each row, we show the masked image (left), reconstruction (centre) and original image (right). A masking ratio of 0.75 was applied to patches of size $16\times16$ on images of size $448\times448$.
• Figure A.3: Training histories for Terra MODIS Vegetation prediction. Datapoints indicated at Epoch 0 are after one epoch of training. Models were not evaluated before the first epoch.