Hierarchical Attention and Parallel Filter Fusion Network for Multi-Source Data Classification

TL;DR

The paper tackles joint hyperspectral and SAR data classification by introducing HAPNet, which combines a Hierarchical Attention Module (HAM) for global, spectral, and local feature modeling with a Parallel Filter Fusion Module (PFFM) that fuses HSI and SAR features in the frequency domain. HAM captures multi-granularity information, while PFFM enhances cross-modal interactions via learnable frequency-domain filters, producing fused features for classification. On Augsburg and Berlin datasets, HAPNet achieves state-of-the-art performance (OA up to 91.44% and 80.51%), validating its effectiveness and robustness, particularly in challenging conditions and for the water class. The approach provides a scalable, frequency-aware fusion framework for multi-source remote sensing classification with practical implications for robust land-cover mapping in cloudy or heterogeneous environments.

Abstract

Hyperspectral image (HSI) and synthetic aperture radar (SAR) data joint classification is a crucial and yet challenging task in the field of remote sensing image interpretation. However, feature modeling in existing methods is deficient to exploit the abundant global, spectral, and local features simultaneously, leading to sub-optimal classification performance. To solve the problem, we propose a hierarchical attention and parallel filter fusion network for multi-source data classification. Concretely, we design a hierarchical attention module for hyperspectral feature extraction. This module integrates global, spectral, and local features simultaneously to provide more comprehensive feature representation. In addition, we develop parallel filter fusion module which enhances cross-modal feature interactions among different spatial locations in the frequency domain. Extensive experiments on two multi-source remote sensing data classification datasets verify the superiority of our proposed method over current state-of-the-art classification approaches. Specifically, our proposed method achieves 91.44% and 80.51% of overall accuracy (OA) on the respective datasets, highlighting its superior performance.

Figures (5)

• Figure 1: Overview of the proposed HAPNet. In the HAPNet Block, multi-level spatial signals are fused through Hierarchical Attention Module (HAM), and the ability to interact and fuse feature information is enhanced through the Parallel Filter Fusion Module (PFFM).
• Figure 2: Illustration of Hierarchical Attention Module (HAM). Global, spectral and local features are modeled in parallel. The global and spectral features are extracted via anchored self-attention in the spatial and spectral dimensions, respectively. The local features captures local structures via depth-wise convolutions.
• Figure 3: Illustration of the Parallel Filter Fusion Network (PFFM).
• Figure 4: Classification results of different methods on the Augsburg dataset.
• Figure 5: Classification results of different methods on the Berlin dataset.