TANet: Triplet Attention Network for All-In-One Adverse Weather Image Restoration

TL;DR

Experimental results show that TANet efficiently and effectively achieves state-of-the-art performance in all-in-one adverse weather image restoration.

Abstract

Adverse weather image restoration aims to remove unwanted degraded artifacts, such as haze, rain, and snow, caused by adverse weather conditions. Existing methods achieve remarkable results for addressing single-weather conditions. However, they face challenges when encountering unpredictable weather conditions, which often happen in real-world scenarios. Although different weather conditions exhibit different degradation patterns, they share common characteristics that are highly related and complementary, such as occlusions caused by degradation patterns, color distortion, and contrast attenuation due to the scattering of atmospheric particles. Therefore, we focus on leveraging common knowledge across multiple weather conditions to restore images in a unified manner. In this paper, we propose a Triplet Attention Network (TANet) to efficiently and effectively address all-in-one adverse weather image restoration. TANet consists of Triplet Attention Block (TAB) that incorporates three types of attention mechanisms: Local Pixel-wise Attention (LPA) and Global Strip-wise Attention (GSA) to address occlusions caused by non-uniform degradation patterns, and Global Distribution Attention (GDA) to address color distortion and contrast attenuation caused by atmospheric phenomena. By leveraging common knowledge shared across different weather conditions, TANet successfully addresses multiple weather conditions in a unified manner. Experimental results show that TANet efficiently and effectively achieves state-of-the-art performance in all-in-one adverse weather image restoration. The source code is available at https://github.com/xhuachris/TANet-ACCV-2024.

Figures (9)

• Figure 1: In TANet, we utilize Triple Attention Block (TAB) to effectively address occlusion and scattering artifacts caused by adverse weather conditions. In TAB, we utilize a Local Pixel-wise Attention (LPA) and a Global Strip-wise Attention (GSA) to address non-uniform degradation patterns. In addition, we utilize Global Distribution Attention to handle unwanted scattering artifacts caused by atmospheric phenomena.
• Figure 2: Architecture of TANet. TANet is an encoder-decoder network comprising several Triplet Attention Blocks (TAB). In TAB, we utilize Local Pixel-wise Attention (LPA), Glbal Strip-wise Attention (GSA), and Global Distribution Attention (GDA) to effectively degradation patterns with occlusion and scattering artifacts. $\textcircled{c}$ and $\oplus$ denote concatenation and addition.
• Figure 3: Architecture of Global Strip-wise Attention (GSA). GSA utilizes horizontal and vertical strip pooling to project features in horizontal and vertical directions. After fusing horizontal and vertical attended features, GSA efficiently addresses degradation patterns with various orientations.
• Figure 4: Qualitative comparison of dehazing performances on the SOTS Resides test set.
• Figure 5: Qualitative comparison of deraining results on the Rain1400 8099669 test set.