Anomaly detection in satellite imagery through temporal inpainting

TL;DR

This work tackles the challenge of detecting surface changes in satellite imagery by reframing change detection as temporal inpainting. A SATLAS-based Swin Transformer predicts the last frame of a Sentinel-2 sequence from preceding frames, with anomalies revealed as reconstruction errors. The method demonstrates strong performance on synthetic perturbations (ROC-AUC up to 0.949 and PR-AUC up to 0.854) and accurately localizes a real-world seismic rupture (Tepehan, 2023), with detection thresholds roughly three times lower than baseline approaches. This approach enables automated, global-scale monitoring using freely available multi-spectral satellite data and a foundation-model framework that generalizes across diverse climates and land covers.

Abstract

Detecting surface changes from satellite imagery is critical for rapid disaster response and environmental monitoring, yet remains challenging due to the complex interplay between atmospheric noise, seasonal variations, and sensor artifacts. Here we show that deep learning can leverage the temporal redundancy of satellite time series to detect anomalies at unprecedented sensitivity, by learning to predict what the surface should look like in the absence of change. We train an inpainting model built upon the SATLAS foundation model to reconstruct the last frame of a Sentinel-2 time series from preceding acquisitions, using globally distributed training data spanning diverse climate zones and land cover types. When applied to regions affected by sudden surface changes, the discrepancy between prediction and observation reveals anomalies that traditional change detection methods miss. We validate our approach on earthquake-triggered surface ruptures from the 2023 Turkey-Syria earthquake sequence, demonstrating detection of a rift feature in Tepehan with higher sensitivity and specificity than temporal median or Reed-Xiaoli anomaly detectors. Our method reaches detection thresholds approximately three times lower than baseline approaches, providing a path towards automated, global-scale monitoring of surface changes from freely available multi-spectral satellite data.

Figures (3)

• Figure 1: Architecture of the temporal inpainting model for anomaly detection.a, Overview of the approach. A sequence of Sentinel-2 acquisitions (frames $t-3$ to $t-1$) along with a partially masked target frame ($t$) are fed to the model, which predicts the complete target. The discrepancy between prediction and observation reveals anomalies. b, Model architecture. The input adapter transforms the multi-temporal, multi-spectral input to match the SATLAS backbone's expected nine channels. The Swin Transformer backbone extracts hierarchical features at five scales. The FPN decoder progressively upsamples and fuses features to produce the final prediction. c, Training strategy. Random masks are applied to the target frame during training, forcing the model to learn temporal relationships. The loss combines L1 reconstruction, structural similarity, and high-frequency detail preservation.
• Figure 2: Anomaly detection performance on synthetic perturbations.a, Receiver operating characteristic (ROC) curves comparing detection methods. Our temporal inpainting approach (green) achieves ROC-AUC of $0.949 \pm 0.104$, outperforming the RX detector (red, $0.904 \pm 0.120$) and temporal median baseline (blue, $0.803 \pm 0.228$). Shaded regions indicate $\pm 1$ standard deviation across test samples. b, Precision-recall curves. Our method achieves PR-AUC of $0.854 \pm 0.233$ compared to $0.711 \pm 0.238$ (RX) and $0.626 \pm 0.304$ (temporal median). The dashed line indicates random classifier performance given the 20% anomaly area fraction. c, Summary of detection metrics across methods. Error bars show standard deviation. d, Detection performance as a function of anomaly intensity. Our method maintains high ROC-AUC even at low intensities where baseline methods approach chance performance, demonstrating superior sensitivity to subtle changes. e, Visual examples of anomaly detection. Columns show ground truth, image with injected anomaly, amplified difference ($\times 5$), inpainting reconstruction error, and ground truth anomaly mask. Cyan contours indicate the inpainting mask boundary.
• Figure 3: Detection of the 2023 Tepehan surface rupture.a, Geographic context. The study area is located near the East Anatolian Fault, which ruptured during the February 6, 2023 earthquake sequence. b, Normalized anomaly score time series for three detection methods. The vertical line indicates the earthquake date. Our method (green) shows the clearest detection with the lowest pre-event background. c, Pre-earthquake and post-earthquake Sentinel-2 images. The surface rupture appears as a dark linear feature through the olive groves. d, Comparison of stable (pre-event) and anomalous (post-event) conditions. Columns show: row label with date, previous frame ($t-1$), target frame ($t$), model residual with anomaly score, and evaluation mask showing valid (green) and excluded (red) regions.