Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Branch Auxiliary Fusion YOLO with Re-parameterization Heterogeneous Convolutional for accurate object detection

Zhiqiang Yang, Qiu Guan, Keer Zhao, Jianmin Yang, Xinli Xu, Haixia Long, Ying Tang

TL;DR

This work addresses the limited ability of traditional PAFPN-based necks to simultaneously preserve shallow, high-resolution details and propagate rich semantic information in real-time detectors. It introduces MAF-YOLO with a plug-and-play Multi-Branch Auxiliary FPN (MAFPN) that combines a Superficial Assisted Fusion (SAF) path for shallow features and an Advanced Assisted Fusion (AAF) path for diversified gradient information, powered by a Global Heterogeneous Kernel Selection (GHKS) mechanism and a Re-parameterized Heterogeneous Efficient Layer Aggregation Network (RepHELAN) using RepHDWConv. The key contributions are the SAF/AAF design for improved multi-scale interaction, the RepHELAN encoder with re-parameterized heterogeneous depthwise convolutions for multi-scale perception, and the GHKS strategy that adaptively expands receptive fields across resolutions; together they yield state-of-the-art real-time performance on COCO with the nano-scale MAF-YOLO achieving 42.4% AP at 3.76M parameters and 10.51G FLOPs, outperforming YOLOv8n by about 5.1% AP. The approach demonstrates strong transferability as a plug-in neck and achieves practical benefits for small-target detection with efficient computation, making it attractive for lightweight deployment scenarios in industry.

Abstract

Due to the effective performance of multi-scale feature fusion, Path Aggregation FPN (PAFPN) is widely employed in YOLO detectors. However, it cannot efficiently and adaptively integrate high-level semantic information with low-level spatial information simultaneously. We propose a new model named MAF-YOLO in this paper, which is a novel object detection framework with a versatile neck named Multi-Branch Auxiliary FPN (MAFPN). Within MAFPN, the Superficial Assisted Fusion (SAF) module is designed to combine the output of the backbone with the neck, preserving an optimal level of shallow information to facilitate subsequent learning. Meanwhile, the Advanced Assisted Fusion (AAF) module deeply embedded within the neck conveys a more diverse range of gradient information to the output layer. Furthermore, our proposed Re-parameterized Heterogeneous Efficient Layer Aggregation Network (RepHELAN) module ensures that both the overall model architecture and convolutional design embrace the utilization of heterogeneous large convolution kernels. Therefore, this guarantees the preservation of information related to small targets while simultaneously achieving the multi-scale receptive field. Finally, taking the nano version of MAF-YOLO for example, it can achieve 42.4% AP on COCO with only 3.76M learnable parameters and 10.51G FLOPs, and approximately outperforms YOLOv8n by about 5.1%. The source code of this work is available at: https://github.com/yang-0201/MAF-YOLO.

Multi-Branch Auxiliary Fusion YOLO with Re-parameterization Heterogeneous Convolutional for accurate object detection

TL;DR

This work addresses the limited ability of traditional PAFPN-based necks to simultaneously preserve shallow, high-resolution details and propagate rich semantic information in real-time detectors. It introduces MAF-YOLO with a plug-and-play Multi-Branch Auxiliary FPN (MAFPN) that combines a Superficial Assisted Fusion (SAF) path for shallow features and an Advanced Assisted Fusion (AAF) path for diversified gradient information, powered by a Global Heterogeneous Kernel Selection (GHKS) mechanism and a Re-parameterized Heterogeneous Efficient Layer Aggregation Network (RepHELAN) using RepHDWConv. The key contributions are the SAF/AAF design for improved multi-scale interaction, the RepHELAN encoder with re-parameterized heterogeneous depthwise convolutions for multi-scale perception, and the GHKS strategy that adaptively expands receptive fields across resolutions; together they yield state-of-the-art real-time performance on COCO with the nano-scale MAF-YOLO achieving 42.4% AP at 3.76M parameters and 10.51G FLOPs, outperforming YOLOv8n by about 5.1% AP. The approach demonstrates strong transferability as a plug-in neck and achieves practical benefits for small-target detection with efficient computation, making it attractive for lightweight deployment scenarios in industry.

Abstract

Due to the effective performance of multi-scale feature fusion, Path Aggregation FPN (PAFPN) is widely employed in YOLO detectors. However, it cannot efficiently and adaptively integrate high-level semantic information with low-level spatial information simultaneously. We propose a new model named MAF-YOLO in this paper, which is a novel object detection framework with a versatile neck named Multi-Branch Auxiliary FPN (MAFPN). Within MAFPN, the Superficial Assisted Fusion (SAF) module is designed to combine the output of the backbone with the neck, preserving an optimal level of shallow information to facilitate subsequent learning. Meanwhile, the Advanced Assisted Fusion (AAF) module deeply embedded within the neck conveys a more diverse range of gradient information to the output layer. Furthermore, our proposed Re-parameterized Heterogeneous Efficient Layer Aggregation Network (RepHELAN) module ensures that both the overall model architecture and convolutional design embrace the utilization of heterogeneous large convolution kernels. Therefore, this guarantees the preservation of information related to small targets while simultaneously achieving the multi-scale receptive field. Finally, taking the nano version of MAF-YOLO for example, it can achieve 42.4% AP on COCO with only 3.76M learnable parameters and 10.51G FLOPs, and approximately outperforms YOLOv8n by about 5.1%. The source code of this work is available at: https://github.com/yang-0201/MAF-YOLO.
Paper Structure (26 sections, 3 equations, 7 figures, 6 tables)

This paper contains 26 sections, 3 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Real-time object detectors
  4. Multi-scale features fusion for object detection
  5. Methodology
  6. Macroscopic architecture
  7. Global Heterogeneous Kernel Selection mechanism
  8. Multi-Branch Auxiliary FPN
  9. Superficial Assisted Fusion.
  10. Advanced Assisted Fusion.
  11. Re-parameterized Heterogeneous Efficient Layer Aggregation Network
  12. Re-parameterized Heterogeneous Depthwise Convolution
  13. Experiments
  14. Experimental Setup
  15. Datasets.
  16. ...and 11 more sections

Figures (7)

  • Figure 1: (a) represents the traditional PAFPN structure, (b) and (c) display the GradCAM++ visualization results of the neck for YOLOv8n and MAF-YOLOn. These three images represent the output layers of the model for small, medium, and large objects.
  • Figure 2: Overview of the network architecture of MAF-YOLO.
  • Figure 3: The architecture of Superficial Assisted Fusion.
  • Figure 4: The architecture of Advanced Assisted Fusion.
  • Figure 5: (a) Overview of the network architecture of RepHELAN, (b) The structure of Inverted Bottleneck in the training and inferencing phases, which is in the RepHELAN, (c) The reparameterization process of a 7×7 RepHDWConv.
  • ...and 2 more figures