Cloud Removal With PolSAR-Optical Data Fusion Using A Two-Flow Residual Network

TL;DR

This paper tackles the problem of cloud-contaminated optical imagery by leveraging PolSAR data through a two-flow PolSAR-Optical fusion network (PODF-CR). The method uses parallel PolSAR and optical encoders, cross-modality fusion (MMCF), and multimodal refinement (MMRF) with attention, while denoising PolSAR via dynamic spatial-channel filters (SCDF) and recovering multi-scale context with ASPP in the decoder. Key contributions include the MMCF and MMRF blocks for effective cross-modal interaction, the SCDF-based denoising of PolSAR features, and the introduction of the OPT-BCFSAR-PFSAR dataset for cloud restoration using polarization features. Experiments show PODF-CR surpasses state-of-the-art methods across PSNR, SSIM, CC, and SAM, with robust texture and structural restoration across varied land-cover and cloud-cover levels, indicating strong potential for large-area, cloud-free optical imaging applications.

Abstract

Optical remote sensing images play a crucial role in the observation of the Earth's surface. However, obtaining complete optical remote sensing images is challenging due to cloud cover. Reconstructing cloud-free optical images has become a major task in recent years. This paper presents a two-flow Polarimetric Synthetic Aperture Radar (PolSAR)-Optical data fusion cloud removal algorithm (PODF-CR), which achieves the reconstruction of missing optical images. PODF-CR consists of an encoding module and a decoding module. The encoding module includes two parallel branches that extract PolSAR image features and optical image features. To address speckle noise in PolSAR images, we introduce dynamic filters in the PolSAR branch for image denoising. To better facilitate the fusion between multimodal optical images and PolSAR images, we propose fusion blocks based on cross-skip connections to enable interaction of multimodal data information. The obtained fusion features are refined through an attention mechanism to provide better conditions for the subsequent decoding of the fused images. In the decoding module, multi-scale convolution is introduced to obtain multi-scale information. Additionally, to better utilize comprehensive scattering information and polarization characteristics to assist in the restoration of optical images, we use a dataset for cloud restoration called OPT-BCFSAR-PFSAR, which includes backscatter coefficient feature images and polarization feature images obtained from PoLSAR data and optical images. Experimental results demonstrate that this method outperforms existing methods in both qualitative and quantitative evaluations.

Figures (13)

• Figure 1: Overview of the proposed PolSAR-Optical data fusion based cloud removal (PODF-CR) algorithm.
• Figure 2: Illustration of the residual blocks based on gated convolutions (RB-GC), the residual blocks based on dynamic filters (RB-DF) and the spatial and channel dynamic filters (SCDF). (a) RB-GC. (b) RB-DF. (c) SCDF.
• Figure 3: Illustration of the multi-modality cross fusion block(MMCF).
• Figure 4: Illustration of the multi-modality refinement fusion. (a) Multi-modal refinement fusion block (MRFB). (b) Spatial-channel attention unit(SCAU).
• Figure 5: Illustration of the atrous spatial pyramid pooling (ASPP) structure.