MHAF-YOLO: Multi-Branch Heterogeneous Auxiliary Fusion YOLO for accurate object detection

TL;DR

This study tackles the challenge of real-time object detection under significant scale variation by enhancing multi-scale feature fusion. It introduces MHAF-YOLO, combining a plug-and-play Multi-Branch Auxiliary FPN (MAFPN) with Superficial Assisted Fusion (SAF) and Advanced Assisted Fusion (AAF), plus Global Adaptive Heterogeneous Flexible Kernel Selection (GHFKS) and Reparameterized Heterogeneous Multi-Scale (RepHMS) modules to expand receptive fields globally and locally. Ablation and cross-dataset experiments demonstrate that these components yield substantial AP gains with strong parameter efficiency, achieving state-of-the-art performance on COCO while also improving VOC segmentation and rotated object detection, and enabling MAFPN to enhance other detectors as well. While inference speed lags behind the fastest YOLO variants due to architectural complexity, MHAF-YOLO presents a versatile foundation for high-precision, real-time detection and motivates future speed optimizations for industrial deployment.

Abstract

Due to the effective multi-scale feature fusion capabilities of the Path Aggregation FPN (PAFPN), it has become a widely adopted component in YOLO-based detectors. However, PAFPN struggles to integrate high-level semantic cues with low-level spatial details, limiting its performance in real-world applications, especially with significant scale variations. In this paper, we propose MHAF-YOLO, a novel detection framework featuring a versatile neck design called the Multi-Branch Auxiliary FPN (MAFPN), which consists of two key modules: the Superficial Assisted Fusion (SAF) and Advanced Assisted Fusion (AAF). The SAF bridges the backbone and the neck by fusing shallow features, effectively transferring crucial low-level spatial information with high fidelity. Meanwhile, the AAF integrates multi-scale feature information at deeper neck layers, delivering richer gradient information to the output layer and further enhancing the model learning capacity. To complement MAFPN, we introduce the Global Heterogeneous Flexible Kernel Selection (GHFKS) mechanism and the Reparameterized Heterogeneous Multi-Scale (RepHMS) module to enhance feature fusion. RepHMS is globally integrated into the network, utilizing GHFKS to select larger convolutional kernels for various feature layers, expanding the vertical receptive field and capturing contextual information across spatial hierarchies. Locally, it optimizes convolution by processing both large and small kernels within the same layer, broadening the lateral receptive field and preserving crucial details for detecting smaller targets. The source code of this work is available at: https://github.com/yang-0201/MHAF-YOLO.

Figures (10)

• Figure 1: Diagram (a) illustrates the PAFPN structure in YOLOv10, while diagrams (b) and (c) display the GradCAM++ visualizations of the feature map results from the same output head for the traditional PAFPN structure and the proposed MAFPN, respectively.
• Figure 2: Comparisons with other state-of-the-art real-time object detectors in terms of parameter-accuracy (left) and GFlops-accuracy (right) trade-offs.
• Figure 3: Different feature extraction blocks. (a)The basic block of CSPNet in yolov5ppyoloeyolov6rtmdet. (b)The proposed block with ELAN in yolov7yolov8yolov9mafyoloyolov10yolo11. (c)The basic unit of YOLOMS yoloms.
• Figure 4: Overview of the network architecture of MHAF-YOLO.
• Figure 5: The architecture of Superficial Assisted Fusion.