FwNet-ECA: A Classification Model Enhancing Window Attention with Global Receptive Fields via Fourier Filtering Operations

TL;DR

FwNet-ECA introduces a Fourier-based Filter Enhancement to create a global receptive field for window attention without shifted windows, complemented by Efficient Channel Attention. By operating in the frequency domain with a learnable weight and employing FFT-driven complexity $O(HW \log(HW))$, the model achieves cross-window information exchange with reduced parameters and computation. Across icartoonface and ImageNet downstream tasks, it attains competitive accuracy while outperforming several shifted-window and baseline architectures in efficiency, with visualization showing broader, more global activations in early layers. The work offers a practical, efficient alternative for visual attention that leverages frequency-domain processing to bridge spatial regions, especially beneficial for fine-grained tasks and scenarios requiring global context.

Abstract

Windowed attention mechanisms were introduced to mitigate the issue of excessive computation inherent in global attention mechanisms. In this paper, we present FwNet-ECA, a novel method that utilizes Fourier transforms paired with learnable weight matrices to enhance the spectral features of images. This method establishes a global receptive field through Filter Enhancement and avoids the use of moving window attention. Additionally, we incorporate the Efficient Channel Attention (ECA) module to improve communication between different channels. Instead of relying on physically shifted windows, our approach leverages frequency domain enhancement to implicitly bridge information across spatial regions. We validate our model on the iCartoonFace dataset and conduct downstream tasks on ImageNet, demonstrating that our model achieves lower parameter counts and computational overheads compared to shifted window approaches, while maintaining competitive accuracy. Furthermore, our visualization operations clearly demonstrated that the Filter Enhancement technique achieves greater effectiveness in the model's shallow layers, where feature maps are relatively larger. This work offers a more efficient and effective alternative for leveraging attention mechanisms in visual processing tasks, alleviating the challenges associated with windowed attention models. Code is available at https://github.com/qingxiaoli/FwNet-ECA

Figures (9)

• Figure 1: Overview of FwNet-ECA Architecture
• Figure 2: Comparison of FwNet-ECA Basic Module vs. Traditional Transformer Basic Module
• Figure 3: Comparison of Model Accuracy on the Flowers-102 Dataset Across Different Computational Costs
• Figure 4: Visual Comparison of Filter Enhancement Operations vs. Shifted Window Attention in the Initial Stage of the Model
• Figure 5: Visual Comparison of Filter Enhancement Operations vs. Shifted Window Attention in the Middle Stage of the Model