Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cross-Layer Feature Self-Attention Module for Multi-Scale Object Detection

Dingzhou Xie, Rushi Lan, Cheng Pang, Enhao Ning, Jiahao Zeng, Wei Zheng

TL;DR

The paper tackles robust multi-scale object detection by explicitly modeling cross-layer dependencies across feature pyramids with a Cross-Layer Feature Self-Attention Module (CFSAM). It introduces a three-component design—local feature extraction, a partition-based Transformer for global cross-layer modeling, and a feature fusion/restoration stage—that enables joint refinement of multi-scale features without heavy upsampling. Empirical results on PASCAL VOC and COCO show substantial improvements over baselines, including VOC mAP of 78.6% and COCO mAP of 52.1%, along with faster convergence and efficient computation via partitioning. The work demonstrates the value of cross-layer attention for integrating fine details and high-level semantics in multi-scale detection, and it presents a plug-and-play approach with attention-based cross-layer modeling.

Abstract

Recent object detection methods have made remarkable progress by leveraging attention mechanisms to improve feature discriminability. However, most existing approaches are confined to refining single-layer or fusing dual-layer features, overlooking the rich inter-layer dependencies across multi-scale representations. This limits their ability to capture comprehensive contextual information essential for detecting objects with large scale variations. In this paper, we propose a novel Cross-Layer Feature Self-Attention Module (CFSAM), which holistically models both local and global dependencies within multi-scale feature maps. CFSAM consists of three key components: a convolutional local feature extractor, a Transformer-based global modeling unit that efficiently captures cross-layer interactions, and a feature fusion mechanism to restore and enhance the original representations. When integrated into the SSD300 framework, CFSAM significantly boosts detection performance, achieving 78.6% mAP on PASCAL VOC (vs. 75.5% baseline) and 52.1% mAP on COCO (vs. 43.1% baseline), outperforming existing attention modules. Moreover, the module accelerates convergence during training without introducing substantial computational overhead. Our work highlights the importance of explicit cross-layer attention modeling in advancing multi-scale object detection.

Cross-Layer Feature Self-Attention Module for Multi-Scale Object Detection

TL;DR

The paper tackles robust multi-scale object detection by explicitly modeling cross-layer dependencies across feature pyramids with a Cross-Layer Feature Self-Attention Module (CFSAM). It introduces a three-component design—local feature extraction, a partition-based Transformer for global cross-layer modeling, and a feature fusion/restoration stage—that enables joint refinement of multi-scale features without heavy upsampling. Empirical results on PASCAL VOC and COCO show substantial improvements over baselines, including VOC mAP of 78.6% and COCO mAP of 52.1%, along with faster convergence and efficient computation via partitioning. The work demonstrates the value of cross-layer attention for integrating fine details and high-level semantics in multi-scale detection, and it presents a plug-and-play approach with attention-based cross-layer modeling.

Abstract

Recent object detection methods have made remarkable progress by leveraging attention mechanisms to improve feature discriminability. However, most existing approaches are confined to refining single-layer or fusing dual-layer features, overlooking the rich inter-layer dependencies across multi-scale representations. This limits their ability to capture comprehensive contextual information essential for detecting objects with large scale variations. In this paper, we propose a novel Cross-Layer Feature Self-Attention Module (CFSAM), which holistically models both local and global dependencies within multi-scale feature maps. CFSAM consists of three key components: a convolutional local feature extractor, a Transformer-based global modeling unit that efficiently captures cross-layer interactions, and a feature fusion mechanism to restore and enhance the original representations. When integrated into the SSD300 framework, CFSAM significantly boosts detection performance, achieving 78.6% mAP on PASCAL VOC (vs. 75.5% baseline) and 52.1% mAP on COCO (vs. 43.1% baseline), outperforming existing attention modules. Moreover, the module accelerates convergence during training without introducing substantial computational overhead. Our work highlights the importance of explicit cross-layer attention modeling in advancing multi-scale object detection.
Paper Structure (12 sections, 5 equations, 5 figures, 6 tables)

This paper contains 12 sections, 5 equations, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Methodology
  4. Local feature extraction
  5. Global feature extraction
  6. Feature fusion restoration
  7. Experiments
  8. Experimental settings
  9. Qualitative comparisons
  10. Quantitative comparisons
  11. Ablation study
  12. Discussion

Figures (5)

  • Figure 1: Illustration of the whole network. By inserting the CFSAM into the SSD network model, the module takes six predicted feature maps as input. After modeling the dependency relationships, the module outputs the same size and number of predicted feature maps, which are used for object prediction.
  • Figure 2: Illustrates the structural diagram of the CFSAM, which includes three parts: local feature extraction, global feature extraction, and feature fusion restoration.
  • Figure 3: Qualitative comparison of the prediction results between our method and other methods on the PASCAL VOC dataset.
  • Figure 4: Qualitative comparison of class activation maps between our method and other methods on the PASCAL VOC dataset.
  • Figure 5: Comparison of training loss between our method and other methods on the PASCAL VOC dataset.