A Complex-valued SAR Foundation Model Based on Physically Inspired Representation Learning

TL;DR

This paper introduces a complex-valued SAR foundation model that embeds physical interpretability by simulating polarimetric decomposition during self-supervised pre-training. It achieves this through scattering queries that represent physically meaningful scattering bases and output corresponding coefficients via a scattering query decoder, guided by polarimetric decomposition and power-consistency losses. The approach yields state-of-the-art performance across six downstream tasks, including complex-valued semantic segmentation, few-shot segmentation, unsupervised classification, and general SAR detection/segmentation, demonstrating strong generalization even with limited labeled data. The framework promises improved interpretability and robustness for SAR interpretation and points toward future multi-modal extensions in remote sensing foundation modeling.

Abstract

Vision foundation models in remote sensing have been extensively studied due to their superior generalization on various downstream tasks. Synthetic Aperture Radar (SAR) offers all-day, all-weather imaging capabilities, providing significant advantages for Earth observation. However, establishing a foundation model for SAR image interpretation inevitably encounters the challenges of insufficient information utilization and poor interpretability. In this paper, we propose a remote sensing foundation model based on complex-valued SAR data, which simulates the polarimetric decomposition process for pre-training, i.e., characterizing pixel scattering intensity as a weighted combination of scattering bases and scattering coefficients, thereby endowing the foundation model with physical interpretability. Specifically, we construct a series of scattering queries, each representing an independent and meaningful scattering basis, which interact with SAR features in the scattering query decoder and output the corresponding scattering coefficient. To guide the pre-training process, polarimetric decomposition loss and power self-supervision loss are constructed. The former aligns the predicted coefficients with Yamaguchi coefficients, while the latter reconstructs power from the predicted coefficients and compares it to the input image's power. The performance of our foundation model is validated on six typical downstream tasks, achieving state-of-the-art results. Notably, the foundation model can extract stable feature representations and exhibits strong generalization, even in data-scarce conditions.

Figures (14)

• Figure 1: The paradigm of SAR foundation model + downstream tasks. The foundation model explores unlabeled SAR data to extract the semantics of interested areas and classes, and the pre-trained foundation model can be broadly applied across various downstream tasks and applications.
• Figure 2: Amplitude and phase information in complex-valued SAR imagery. The radar sensor transmits electromagnetic (EM) waves and receives complex-valued radar signals. The amplitude image highlights strong scattering points and clear contours, while the phase image contains more detailed information about the structure of objects, such as aircraft.
• Figure 3: Comparison of previous SAR-based foundation model and ours. (a) Pevious SAR-based foundation models use amplitude images as input and reconstruct pixel values through masked image modelingCROMA. (b) Our foundation model takes amplitude and phase images as input and reconstructs power values by simulating the physical process of SAR polarimetric decomposition.
• Figure 4: Schematic diagram of the Yamaguchi decomposition components. The top line exhibits fully PolSAR images, arranged from left to right with polarization modes HH, HV, VH, and VV respectively. The subsequent line depicts the results of four components from $T$, illustrating Dbl scattering, Hlx scattering, S.F. scattering, and Vol scattering components respectively.
• Figure 5: Overview of our foundation model based on complex-valued SAR images. Deep network simulates the physical process of polarimetric decomposition, constrained by the polarimetric decomposition loss and power self-supervised loss during the pre-training phase. The embedding and enhancement of the physical meaning of the scattering bases is achieved through scattering query construction.