WaveMAE: Wavelet decomposition Masked Auto-Encoder for Remote Sensing

TL;DR

WaveMAE addresses data scarcity in remote sensing by combining a Discrete Wavelet Transform (DWT) with masked autoencoding to learn scale-aware, frequency-separated representations for multispectral imagery. It introduces a Geo-conditioned Positional Encoding (GPE) based on Spherical Harmonics to inject geographic priors into the Transformer encoder, while using a multi-level patch embedding and tube masking to align wavelet components across scales. Pretraining on fMoW-S2 and evaluation on the PANGAEA benchmark demonstrate consistent gains over previous masked autoencoder baselines, with a lightweight WaveMAE-Small achieving state-of-the-art performance with only $26.4\%$ of the parameters. Ablation studies confirm the importance of decomposition depth, GPE regularization, masking ratio, and token size, establishing WaveMAE as a geography-aware, frequency-aware foundation model for optical remote sensing. These results suggest practical impact for improved segmentation, regression, and change-detection tasks in large-scale RS pipelines and inspire future multimodal and temporal extensions.

Abstract

Self-supervised learning (SSL) has recently emerged as a key strategy for building foundation models in remote sensing, where the scarcity of annotated data limits the applicability of fully supervised approaches. In this work, we introduce WaveMAE, a masked autoencoding framework tailored for multispectral satellite imagery. Unlike conventional pixel-based reconstruction, WaveMAE leverages a multi-level Discrete Wavelet Transform (DWT) to disentangle frequency components and guide the encoder toward learning scale-aware high-frequency representations. We further propose a Geo-conditioned Positional Encoding (GPE), which incorporates geographical priors via Spherical Harmonics, encouraging embeddings that respect both semantic and geospatial structure. To ensure fairness in evaluation, all methods are pretrained on the same dataset (fMoW-S2) and systematically evaluated on the diverse downstream tasks of the PANGAEA benchmark, spanning semantic segmentation, regression, change detection, and multilabel classification. Extensive experiments demonstrate that WaveMAE achieves consistent improvements over prior state-of-the-art approaches, with substantial gains on segmentation and regression benchmarks. The effectiveness of WaveMAE pretraining is further demonstrated by showing that even a lightweight variant, containing only 26.4% of the parameters, achieves state-of-the-art performance. Our results establish WaveMAE as a strong and geographically informed foundation model for multispectral remote sensing imagery.

Figures (10)

• Figure 1: Representation of a 3 level 2D Discrete Wavelet Transform applied on an image. The decomposition from pixel values highlighted in red generates the wavelet components LL (low frequencies), LH (vertical high frequencies), HL (horizontal high frequencies), and HH (diagonal high frequencies) on different scales.
• Figure 2: Illustration of WaveMAE architecture, the core design follows that of the MAE mae_paper. We apply a 4-level DWT decomposition to the input image $x$ of resolution $(H,W)$ is obtaining 4 set of wavelet components at different scales of resolutions $(H/2, W/2), (H/4, W/4), (H/8, W/8)$ and $(H/16, W/16)$. Following this operation the components in $S$ are fed to the Multi-level patch embedding, here lighter colors stand for shallow decomposition levels and low-scale feature extraction, on the opposite darker colors mean deeper decomposition level and high-scale feature extraction. After, the full sequence is fed to the MAE framework, the decoder reconstructs the masked wavelet components $\bar{S}$ at the same resolution specified previously. A smooth L1 loss is calculated between the input and the predicted wavelet components. Additionally, the predicted components are used to calculate the inverse DWT, obtaining the reconstructed multispectral image $\bar{x}$. An MSE loss is calculated between the latter and the input image $x$. In both losses we attend only to the previously masked parts of the components/reconstructed image for the calculation.
• Figure 3: In this figure we show the mechanism of our Multi-level Patch Embedding. Considering an input image of spatial dimension $64\times 64$, for $N=4$ decomposition level we obtain the set of DWT components of resolution specified in the table. Scaling the patch size accordingly to the component's spatial resolution, allows us to attend to the same information of the input image in the correspondent spatial location at different decomposition levels. White color is used to show the grid of pixels while yellow outlines the patches borders.
• Figure 4: The illustration shows how we obtain the novel Geo-conditioned Positional Encoding. Firstly, from any image $x$ the metadata associated are read to extract the geospatial coordinates in terms of latitude and longitude. Since coordinates are stored in degree we apply a degree to radians conversion and calculate a set of Spherical Harmonics as specified in Section \ref{['subsection:pe']}. After this step we apply a linear projection to bring the encoding to the embedding dimension $D=768$ of our encoder.
• Figure 5: Tube Masking strategy: supposing a 3 level DWT decomposition each component in all levels are masked spatially in the same way. From top to bottom, in each row, we have shallow to deeper level of decomposition, components are indicated in different colors [LL, HL, LH, HH], gray patches are masked.