Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LEFormer: A Hybrid CNN-Transformer Architecture for Accurate Lake Extraction from Remote Sensing Imagery

Ben Chen, Xuechao Zou, Yu Zhang, Jiayu Li, Kai Li, Junliang Xing, Pin Tao

TL;DR

This work tackles accurate lake extraction from remote sensing imagery, addressing boundary blur and noise with a hybrid CNN-Transformer architecture, LEFormer. LEFormer comprises a CNN encoder for local detail, a lightweight Transformer encoder for global context, and a Cross-Encoder Fusion to integrate features before decoding. On SW and QTPL datasets, LEFormer achieves state-of-the-art mIoU with 3.61M parameters and substantially lower FLOPs than prior methods, e.g., 1.27G FLOPs on SW and about 48x fewer FLOPs than MSNANet, while maintaining high accuracy. The results demonstrate the effectiveness of combining multi-scale local features with global self-attention in a lightweight fusion framework for robust lake masks in large-scale remote sensing tasks.

Abstract

Lake extraction from remote sensing images is challenging due to the complex lake shapes and inherent data noises. Existing methods suffer from blurred segmentation boundaries and poor foreground modeling. This paper proposes a hybrid CNN-Transformer architecture, called LEFormer, for accurate lake extraction. LEFormer contains three main modules: CNN encoder, Transformer encoder, and cross-encoder fusion. The CNN encoder effectively recovers local spatial information and improves fine-scale details. Simultaneously, the Transformer encoder captures long-range dependencies between sequences of any length, allowing them to obtain global features and context information. The cross-encoder fusion module integrates the local and global features to improve mask prediction. Experimental results show that LEFormer consistently achieves state-of-the-art performance and efficiency on the Surface Water and the Qinghai-Tibet Plateau Lake datasets. Specifically, LEFormer achieves 90.86% and 97.42% mIoU on two datasets with a parameter count of 3.61M, respectively, while being 20 minor than the previous best lake extraction method. The source code is available at https://github.com/BastianChen/LEFormer.

LEFormer: A Hybrid CNN-Transformer Architecture for Accurate Lake Extraction from Remote Sensing Imagery

TL;DR

This work tackles accurate lake extraction from remote sensing imagery, addressing boundary blur and noise with a hybrid CNN-Transformer architecture, LEFormer. LEFormer comprises a CNN encoder for local detail, a lightweight Transformer encoder for global context, and a Cross-Encoder Fusion to integrate features before decoding. On SW and QTPL datasets, LEFormer achieves state-of-the-art mIoU with 3.61M parameters and substantially lower FLOPs than prior methods, e.g., 1.27G FLOPs on SW and about 48x fewer FLOPs than MSNANet, while maintaining high accuracy. The results demonstrate the effectiveness of combining multi-scale local features with global self-attention in a lightweight fusion framework for robust lake masks in large-scale remote sensing tasks.

Abstract

Lake extraction from remote sensing images is challenging due to the complex lake shapes and inherent data noises. Existing methods suffer from blurred segmentation boundaries and poor foreground modeling. This paper proposes a hybrid CNN-Transformer architecture, called LEFormer, for accurate lake extraction. LEFormer contains three main modules: CNN encoder, Transformer encoder, and cross-encoder fusion. The CNN encoder effectively recovers local spatial information and improves fine-scale details. Simultaneously, the Transformer encoder captures long-range dependencies between sequences of any length, allowing them to obtain global features and context information. The cross-encoder fusion module integrates the local and global features to improve mask prediction. Experimental results show that LEFormer consistently achieves state-of-the-art performance and efficiency on the Surface Water and the Qinghai-Tibet Plateau Lake datasets. Specifically, LEFormer achieves 90.86% and 97.42% mIoU on two datasets with a parameter count of 3.61M, respectively, while being 20 minor than the previous best lake extraction method. The source code is available at https://github.com/BastianChen/LEFormer.
Paper Structure (11 sections, 2 equations, 5 figures, 4 tables)

This paper contains 11 sections, 2 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. METHODOLOGY
  3. CNN Encoder (CE)
  4. Transformer Encoder (TE)
  5. Cross-Encoder Fusion (CEF)
  6. Experiments
  7. Experimental Settings
  8. Ablation Studies
  9. Comparison to the State-of-the-Arts
  10. Conclusion
  11. Acknowledgements

Figures (5)

  • Figure 1: Performance and efficiency comparison on Surface Water dataset. The circle area indicates the number of parameters.
  • Figure 2: Overview architecture of our proposed LEFormer.
  • Figure 3: The diagram depicts the structural layout of the MSCA.
  • Figure 4: The diagram depicts the structural layout of three critical modules: (a) the Efficient Transformer Layer (ETL), (b) the Pooling Transformer Layer (PTL), and (c) the Cross-Encoder Fusion (CEF).
  • Figure 5: Visualization results of our proposed LEFormer and other methods. The white circles indicate apparent differences.