Attentive Contextual Attention for Cloud Removal

TL;DR

This work introduces a novel approach named Attentive Contextual Attention (AC-Attention), which enhances conventional attention mechanisms by dynamically learning data-driven attentive selection scores, enabling it to filter out noise and irrelevant features effectively in the cloud removal framework.

Abstract

Cloud cover can significantly hinder the use of remote sensing images for Earth observation, prompting urgent advancements in cloud removal technology. Recently, deep learning strategies have shown strong potential in restoring cloud-obscured areas. These methods utilize convolution to extract intricate local features and attention mechanisms to gather long-range information, improving the overall comprehension of the scene. However, a common drawback of these approaches is that the resulting images often suffer from blurriness, artifacts, and inconsistencies. This is partly because attention mechanisms apply weights to all features based on generalized similarity scores, which can inadvertently introduce noise and irrelevant details from cloud-covered areas. To overcome this limitation and better capture relevant distant context, we introduce a novel approach named Attentive Contextual Attention (AC-Attention). This method enhances conventional attention mechanisms by dynamically learning data-driven attentive selection scores, enabling it to filter out noise and irrelevant features effectively. By integrating the AC-Attention module into the DSen2-CR cloud removal framework, we significantly improve the model's ability to capture essential distant information, leading to more effective cloud removal. Our extensive evaluation of various datasets shows that our method outperforms existing ones regarding image reconstruction quality. Additionally, we conducted ablation studies by integrating AC-Attention into multiple existing methods and widely used network architectures. These studies demonstrate the effectiveness and adaptability of AC-Attention and reveal its ability to focus on relevant features, thereby improving the overall performance of the networks. The code is available at \url{https://github.com/huangwenwenlili/ACA-CRNet}.

Figures (7)

• Figure 1: A comparison between the Contextual Attention (CA) and our proposed Attentive Contextual Attention (AC-Attention). The first two columns show a selected query feature, marked by red stars, along with features within the top 5% similarity to the query feature, marked by magenta pentagons. The following two columns display the similarity scores for this query feature. The next two columns present the cloud removal results.
• Figure 2: The architecture of Attentive Contextual Attention (AC-Attention), consisting of three steps: embedding and reshaping, attentive matching, and attending.
• Figure 3: Overview of our proposed ACA-CRNet, including (a) ACA-CRNet architecture, (b) Stem, (c) Residual Block (RB), (d) Residual AC-Attention Block (RACAB), and (e) Refine component. The ACA-CRNet is designed in a residual style, comprising the Stem, RBs, RACABs, and Refine modules.
• Figure 4: Visualization of cloud removal results on RICE-I and RICE-II datasets. Local details are highlighted in red boxes. Zooming in is recommended for a clearer view.
• Figure 5: Visualization of cloud removal results for Sentinel-2 satellite data on the SEN12MS-CR dataset. The Sentinel-2 data includes 13 spectral bands, with the visualization images generated using the RGB bands. Zooming in is recommended for a clearer view.