RFAConv: Innovating Spatial Attention and Standard Convolutional Operation

TL;DR

The paper identifies a fundamental limitation of standard convolution: parameter sharing across spatial positions constrains large-kernel expressiveness. It introduces Receptive-Field Attention (RFA) and the RFAConv operation, which focus attention on receptive-field spatial features to produce per-receptive-field attention without substantial cost. Variants RFCBAMConv and RFCAConv extend this framework by integrating refined channel-spatial attention to capture long-range information, yielding improved results on ImageNet-1k, COCO, and VOC benchmarks with only modest increases in parameters and FLOPs. Visualizations and extensive experiments support the claim that directing spatial attention to receptive-field features enhances feature discrimination and localization, suggesting a practical path toward more powerful attention-augmented convolutions. The approach offers a balanced trade-off between performance gains and efficiency, encouraging broader adoption of receptive-field–oriented attention in CNNs.

Abstract

Spatial attention has been widely used to improve the performance of convolutional neural networks. However, it has certain limitations. In this paper, we propose a new perspective on the effectiveness of spatial attention, which is that the spatial attention mechanism essentially solves the problem of convolutional kernel parameter sharing. However, the information contained in the attention map generated by spatial attention is not sufficient for large-size convolutional kernels. Therefore, we propose a novel attention mechanism called Receptive-Field Attention (RFA). Existing spatial attention, such as Convolutional Block Attention Module (CBAM) and Coordinated Attention (CA) focus only on spatial features, which does not fully address the problem of convolutional kernel parameter sharing. In contrast, RFA not only focuses on the receptive-field spatial feature but also provides effective attention weights for large-size convolutional kernels. The Receptive-Field Attention convolutional operation (RFAConv), developed by RFA, represents a new approach to replace the standard convolution operation. It offers nearly negligible increment of computational cost and parameters, while significantly improving network performance. We conducted a series of experiments on ImageNet-1k, COCO, and VOC datasets to demonstrate the superiority of our approach. Of particular importance, we believe that it is time to shift focus from spatial features to receptive-field spatial features for current spatial attention mechanisms. In this way, we can further improve network performance and achieve even better results. The code and pre-trained models for the relevant tasks can be found at https://github.com/Liuchen1997/RFAConv.

Figures (11)

• Figure 1: It simply represents a 3$\times$3 convolution operation. The features are obtained by multiplying the convolution kernel with a receptive-field slider of the same size and then summing.
• Figure 2: The original feature map highlights the key features by learned attention map. This process of highlighting is the Re-weight ($\times$) operation.
• Figure 3: The convolutional kernel parameter $K_{i}$ obtained by multiplying the attentional weight $A_{i}$ with the convolutional kernel parameter K is different in each receptive-field slider, i.e., $Kernel 1\ne Kernel 2\ne Kernel 3\ne...\ne Kernel N$.
• Figure 4: It is obvious that there is an overlap of features in each receptive-field slider, which leads to the problem of sharing of attentional weights across sliders.
• Figure 5: The receptive-field spatial features are obtained by transforming the spatial features.