Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AMMUNet: Multi-Scale Attention Map Merging for Remote Sensing Image Segmentation

Yang Yang, Shunyi Zheng

TL;DR

AMMUNet addresses the challenge of capturing global context in remote sensing semantic segmentation without prohibitive computation. It introduces Granular Multi-head Self-Attention (GMSA) and Attention Map Merging Mechanism (AMMM) within a UNet framework that uses a ResNet encoder, enabling efficient global modeling through multi-scale attention map fusion. The GMSA provides fine-grained local attention while AMMM merges multi-scale maps with a fixed mask to realize global dependencies at reduced cost, aided by the Dimension Correspondence Module for scale alignment. On Vaihingen and Potsdam, AMMUNet achieves state-of-the-art $mIoU$ scores of $75.48\%$ and $77.90\%$ respectively, with strong $mAcc$ performance and favorable efficiency, underscoring its practical value for high-resolution remote sensing segmentation.

Abstract

The advancement of deep learning has driven notable progress in remote sensing semantic segmentation. Attention mechanisms, while enabling global modeling and utilizing contextual information, face challenges of high computational costs and require window-based operations that weaken capturing long-range dependencies, hindering their effectiveness for remote sensing image processing. In this letter, we propose AMMUNet, a UNet-based framework that employs multi-scale attention map merging, comprising two key innovations: the granular multi-head self-attention (GMSA) module and the attention map merging mechanism (AMMM). GMSA efficiently acquires global information while substantially mitigating computational costs in contrast to global multi-head self-attention mechanism. This is accomplished through the strategic utilization of dimension correspondence to align granularity and the reduction of relative position bias parameters, thereby optimizing computational efficiency. The proposed AMMM effectively combines multi-scale attention maps into a unified representation using a fixed mask template, enabling the modeling of global attention mechanism. Experimental evaluations highlight the superior performance of our approach, achieving remarkable mean intersection over union (mIoU) scores of 75.48\% on the challenging Vaihingen dataset and an exceptional 77.90\% on the Potsdam dataset, demonstrating the superiority of our method in precise remote sensing semantic segmentation. Codes are available at https://github.com/interpretty/AMMUNet.

AMMUNet: Multi-Scale Attention Map Merging for Remote Sensing Image Segmentation

TL;DR

AMMUNet addresses the challenge of capturing global context in remote sensing semantic segmentation without prohibitive computation. It introduces Granular Multi-head Self-Attention (GMSA) and Attention Map Merging Mechanism (AMMM) within a UNet framework that uses a ResNet encoder, enabling efficient global modeling through multi-scale attention map fusion. The GMSA provides fine-grained local attention while AMMM merges multi-scale maps with a fixed mask to realize global dependencies at reduced cost, aided by the Dimension Correspondence Module for scale alignment. On Vaihingen and Potsdam, AMMUNet achieves state-of-the-art scores of and respectively, with strong performance and favorable efficiency, underscoring its practical value for high-resolution remote sensing segmentation.

Abstract

The advancement of deep learning has driven notable progress in remote sensing semantic segmentation. Attention mechanisms, while enabling global modeling and utilizing contextual information, face challenges of high computational costs and require window-based operations that weaken capturing long-range dependencies, hindering their effectiveness for remote sensing image processing. In this letter, we propose AMMUNet, a UNet-based framework that employs multi-scale attention map merging, comprising two key innovations: the granular multi-head self-attention (GMSA) module and the attention map merging mechanism (AMMM). GMSA efficiently acquires global information while substantially mitigating computational costs in contrast to global multi-head self-attention mechanism. This is accomplished through the strategic utilization of dimension correspondence to align granularity and the reduction of relative position bias parameters, thereby optimizing computational efficiency. The proposed AMMM effectively combines multi-scale attention maps into a unified representation using a fixed mask template, enabling the modeling of global attention mechanism. Experimental evaluations highlight the superior performance of our approach, achieving remarkable mean intersection over union (mIoU) scores of 75.48\% on the challenging Vaihingen dataset and an exceptional 77.90\% on the Potsdam dataset, demonstrating the superiority of our method in precise remote sensing semantic segmentation. Codes are available at https://github.com/interpretty/AMMUNet.
Paper Structure (14 sections, 9 equations, 3 figures, 3 tables)

This paper contains 14 sections, 9 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. METHODOLOGY
  3. Overview of AMMUNet
  4. Global Multi-head Self-Attention
  5. Granular Multi-head Self-Attention
  6. Attention Map Merging Mechanism
  7. Dimension Correspondence Module
  8. EXPERIMENT
  9. Dataset
  10. Evaluation Metrics
  11. Implementation Details
  12. Comparison with State-of-the-Art Methods
  13. Ablation Study
  14. CONCLUSION

Figures (3)

  • Figure 1: (a) Overview of AMMUnet. (b) Detailed structure of GMSA. (c) Detailed structure of AMMM. (d) Detailed structure of MSA.
  • Figure 2: Visualization results on the vaihingen dataset.
  • Figure 3: Visualization results on the potsdam dataset.