PAD: Phase-Amplitude Decoupling Fusion for Multi-Modal Land Cover Classification

TL;DR

This work tackles the challenge of fusing SAR and RGB data for land-cover classification by revealing that phase captures modality-shared structure while amplitude encodes modality-complementary details in the Fourier domain. It introduces PAD, a frequency-aware fusion framework with Phase Spectrum Correction (PSC) and Amplitude Spectrum Fusion (ASF) that decouple and then reintegrate phase and amplitude representations, guided by two spectral priors. The method demonstrates state-of-the-art performance on WHU-OPT-SAR and DDHR-SK, with strong ablation results showing the complementary and synergistic roles of PSC and ASF, as well as robustness under missing data and cloud scenarios. The approach offers a physics-informed paradigm for multi-modal remote sensing fusion and provides data-efficient, real-time capable fusion suitable for operational deployment, with code to be released publicly.

Abstract

The fusion of Synthetic Aperture Radar (SAR) and RGB imagery for land cover classification remains challenging due to modality heterogeneity and underexploited spectral complementarity. Existing approaches often fail to decouple shared structural features from modality-complementary radiometric attributes, resulting in feature conflicts and information loss. To address this, we propose Phase-Amplitude Decoupling (PAD), a frequency-aware framework that separates phase (modality-shared) and amplitude (modality-complementary) components in the Fourier domain. This design reinforces shared structures while preserving complementary characteristics, thereby enhancing fusion quality. Unlike previous methods that overlook the distinct physical properties encoded in frequency spectra, PAD explicitly introduces amplitude-phase decoupling for multi-modal fusion. Specifically, PAD comprises two key components: 1) Phase Spectrum Correction (PSC), which aligns cross-modal phase features via convolution-guided scaling to improve geometric consistency; and 2) Amplitude Spectrum Fusion (ASF), which dynamically integrates high- and low-frequency patterns using frequency-adaptive multilayer perceptrons, effectively exploiting SAR's morphological sensitivity and RGB's spectral richness. Extensive experiments on WHU-OPT-SAR and DDHR-SK demonstrate state-of-the-art performance. This work establishes a new paradigm for physics-aware multi-modal fusion in remote sensing. The code will be available at https://github.com/RanFeng2/PAD.

Figures (8)

• Figure 1: Motivation and SAR-RGB Spatial-Frequency Difference Analysis. (a) SAR and RGB imagery exhibit modality gaps due to distinct sensing mechanisms (backscattering vs. reflectance). (b) The fusion feature space contains heterogeneous, shared, and complementary components; effective fusion suppresses heterogeneous cues while enhancing shared and complementary patterns. (c-g) Frequency-domain analysis on WHU-OPT-SAR (see §\ref{['sec:iii_a']}): (c) spatial discrepancies show anisotropic distributions; (d) relative amplitude differences concentrate in high-frequency regions; (e) phase discrepancies are minimal and primarily cluster in low-frequency bands; (f-g) varying the sampling rate significantly affects spatial discrepancy distributions.
• Figure 2: Flowchart of the PAD Framework. Registered RGB and SAR inputs, $I_{\mathrm{RGB}}$ and $I_{\mathrm{SAR}}$, are processed by asymmetric backbones to extract features $x_{\mathrm{RGB}}$ and $x_{\mathrm{SAR}}$ at each stage. These features pass through $n$ PAD fusion modules, each comprising: (a) SCF; (b) FD (spatial-to-frequency transform); (c) ASF; (d) PSC; and (e) FR (frequency-to-spatial transform). At each stage, the fused features are concatenated channel-wise (outer circle, $C$) and fed into a shared decoder. Here, $\mathcal{A}$ and $\mathcal{P}$ denote the amplitude and phase spectra, respectively.
• Figure 3: Statistical Analysis of APPD. (a) APPD spectrum. (b-d) Frequency histograms with Gaussian fits for low frequency (LF; radius = $0.5\times$ half-diagonal), high frequency (HF), and all spectra (ALL). (e-f) Mean-variance trajectories across sampling rates. Across sampling rates, all spectra exhibit near-Gaussian behavior, while slight deviations in LF and ALL are attributable to structural noise at low frequencies, indicating that the phase, particularly HF, serves as a robust shared feature.
• Figure 4: Visualization of Spatial-Frequency Spectrum Differences under Missing RGB Regions. (a-b) Two RGB images have missing regions in the original dataset. (c) Variation in RSD, RAD, and APPD, indicating RSD’s greater sensitivity to missing data. (d-f) Difference spectra computed including the missing regions. (g-i) Difference spectra computed excluding the missing regions. Overall, frequency-domain metrics (RAD and APPD) are more robust to missing data than the spatial-domain RSD.
• Figure 5: Visualization of Spatial-Frequency Spectrum Differences under Cloud Obscuration. (a-c) Example observations with cloud masks derived from the RGB, SAR, and NIR channels in WHU-OPT-SARwright2025cloud, respectively. (d) Comparison of RSD, RAD, and APPD between cloud-covered and cloud-free areas; arrows indicate the value of the decrease.