Pansharpening for Thin-Cloud Contaminated Remote Sensing Images: A Unified Framework and Benchmark Dataset

Abstract

Pansharpening under thin cloudy conditions is a practically significant yet rarely addressed task, challenged by simultaneous spatial resolution degradation and cloud-induced spectral distortions. Existing methods often address cloud removal and pansharpening sequentially, leading to cumulative errors and suboptimal performance due to the lack of joint degradation modeling. To address these challenges, we propose a Unified Pansharpening Model with Thin Cloud Removal (Pan-TCR), an end-to-end framework that integrates physical priors. Motivated by theoretical analysis in the frequency domain, we design a frequency-decoupled restoration (FDR) block that disentangles the restoration of multispectral image (MSI) features into amplitude and phase components, each guided by complementary degradation-robust prompts: the near-infrared (NIR) band amplitude for cloud-resilient restoration, and the panchromatic (PAN) phase for high-resolution structural enhancement. To ensure coherence between the two components, we further introduce an interactive inter-frequency consistency (IFC) module, enabling cross-modal refinement that enforces consistency and robustness across frequency cues. Furthermore, we introduce the first real-world thin-cloud contaminated pansharpening dataset (PanTCR-GF2), comprising paired clean and cloudy PAN-MSI images, to enable robust benchmarking under realistic conditions. Extensive experiments on real-world and synthetic datasets demonstrate the superiority and robustness of Pan-TCR, establishing a new benchmark for pansharpening under realistic atmospheric degradations.

Figures (5)

• Figure 1: Collection of the first thin-cloud pansharpening dataset (a) and comparison of processing paradigms and performance between existing methods (b) and proposed approach (c). ${\color{first_Blue}}$: PSNR, ${\color{first_Orange}}$: Parameters, ${\color{first_Yellow}}$: Test Time, ${\color{first_Green}}$: FLOPs.
• Figure 2: Motivations. (a) Spectral sensitivity to thin-cloud across different wavelengths $\lambda$, modeled via atmospheric scattering model $E(\lambda)$, where $E_0(\lambda)$, $L_\infty(\lambda)$ and $e^{-k(\lambda) d}$ denote clean surface radiance, airlight and transmittance, respectively, and $1-e^{-k(\lambda) d}$ denotes cloud interference. NIR band is less affected by cloud scattering than shorter wavelengths (e.g., B, G, R). Although NIR is cloud-robust, its modality difference from visible bands presents challenges for spectral consistency. (b) The amplitude component primarily reflects global intensity distributions and is highly sensitive to cloud-induced degradation, whereas (c) the phase component encodes fine structural details and is more affected by resolution degradation. Moreover, directly exchanging amplitude or phase between images can result in inter-frequency inconsistency artifacts (highlighted by yellow circles).
• Figure 3: Overview of proposed framework. The left panel illustrates the overall architecture, which takes a stacked input of LR-MSI and PAN images and progressively reconstructs cloud-free high-resolution MSI (HR-MSI) outputs. The central panel illustrates the core frequency-decoupled restoration (FDR) block, which incorporates key sub-modules, including modality-aware frequency gating (MAFG) and interactive inter-frequency consistency (IFC) to guide amplitude and phase restoration via degradation-robust frequency prompts. The right panel provides the symbols used throughout the pipeline.
• Figure 4: Overview of PanTCR-GF2 dataset: Distribution of land cover types with representative paired examples.
• Figure 5: Qualitative results on PanTCR-GF2 and WV3 datasets with reduced-resolution, including restored images and residual error maps. Inputs (“LR-MSI", “PAN") or methods marked with “*" indicate two-stage pipelines, where cloud removal is performed prior to pansharpening.