TerraCodec: Compressing Earth Observations

TL;DR

TerraCodec addresses the escalating storage and transmission demands of Earth observation data by introducing a family of learned codecs tailored to multispectral and temporally evolving EO imagery. The approach combines image-focused codecs (TEC-FP, TEC-ELIC) with a temporal transformer (TEC-TT) and a flexible-rate framework (FlexTEC) enabled by Latent Repacking, trained on large Sentinel-2 data and evaluated against classical codecs. Key findings show 3–10x compression gains at equivalent image quality and strong zero-shot cloud inpainting capabilities, underscoring the practical value of learned, EO-specific priors. The work provides a foundation for high-performance, domain-aware EO compression and demonstrates positive impact on downstream tasks, with code and pretrained weights to be released publicly.

Abstract

Earth observation (EO) satellites produce massive streams of multispectral image time series, posing pressing challenges for storage and transmission. Yet, learned EO compression remains fragmented, lacking publicly available pretrained models and misaligned with advances in compression for natural imagery. Image codecs overlook temporal redundancy, while video codecs rely on motion priors that fail to capture the radiometric evolution of largely static scenes. We introduce TerraCodec (TEC), a family of learned codecs tailored to EO. TEC includes efficient image-based variants adapted to multispectral inputs, as well as a Temporal Transformer model (TEC-TT) that leverages dependencies across time. To overcome the fixed-rate setting of today's neural codecs, we present Latent Repacking, a novel method for training flexible-rate transformer models that operate on varying rate-distortion settings. Trained on Sentinel-2 data, TerraCodec outperforms classical codecs, achieving 3-10x stronger compression at equivalent image quality. Beyond compression, TEC-TT enables zero-shot cloud inpainting, surpassing state-of-the-art methods on the AllClear benchmark. Our results establish bespoke, learned compression algorithms as a promising direction for Earth observation. Code and model weights will be released under a permissive license.

Figures (20)

• Figure 1: Varying reconstruction quality at a similar compression rate.
• Figure 1: Test PSNR on AllClear across difficulty subsets (by average cloudiness). PSNR computed across all bands following AllClear.
• Figure 2: TerraCodec image codecs. Factorized Prior balle2017factorized uses a fully factorized prior without context, and assumes a zero-centered normal distribution for the latents. ELIC he2022elic augments a hyperprior with spatial and channel context to predict per-latent mean/scale.
• Figure 3: Architecture of the TerraCodec-TT model following Mentzer2022VCT. Each image is encoded with ELIC followed by a temporal transformer that predicts mean and scale for new images.
• Figure 4: Latent Repacking converts $T$ spatial tokens $(W \cdot H)$ into channel-slice tokens so each token carries scene-wide content. During training, we sample a token budget $K$, mask the rest using a learned token m, and scale the rate. During inference, the user can pick the compression level $K$.