SFFNet: Synergistic Feature Fusion Network With Dual-Domain Edge Enhancement for UAV Image Object Detection

Abstract

Object detection in unmanned aerial vehicle (UAV) images remains a highly challenging task, primarily caused by the complexity of background noise and the imbalance of target scales. Traditional methods easily struggle to effectively separate objects from intricate backgrounds and fail to fully leverage the rich multi-scale information contained within images. To address these issues, we have developed a synergistic feature fusion network (SFFNet) with dual-domain edge enhancement specifically tailored for object detection in UAV images. Firstly, the multi-scale dynamic dual-domain coupling (MDDC) module is designed. This component introduces a dual-driven edge extraction architecture that operates in both the frequency and spatial domains, enabling effective decoupling of multi-scale object edges from background noise. Secondly, to further enhance the representation capability of the model's neck in terms of both geometric and semantic information, a synergistic feature pyramid network (SFPN) is proposed. SFPN leverages linear deformable convolutions to adaptively capture irregular object shapes and establishes long-range contextual associations around targets through the designed wide-area perception module (WPM). Moreover, to adapt to the various applications or resource-constrained scenarios, six detectors of different scales (N/S/M/B/L/X) are designed. Experiments on two challenging aerial datasets (VisDrone and UAVDT) demonstrate the outstanding performance of SFFNet-X, achieving 36.8 AP and 20.6 AP, respectively. The lightweight models (N/S) also maintain a balance between detection accuracy and parameter efficiency. The code will be available at https://github.com/CQNU-ZhangLab/SFFNet.

Figures (7)

• Figure 1: The relationship between the AP value and the number of parameters for different object detection algorithms on the VisDrone dataset. The points near the upper left corner of the figure indicate that the model can achieve higher accuracy while maintaining a lower number of parameters. Our algorithm (marked “Ours”) outperforms others by achieving a higher AP value with fewer parameters.
• Figure 2: Overview of the SFFNet framework. The framework integrates a backbone network with MDDC for efficient multi-scale dual-domain feature extraction, and realizes collaborative feature fusion in the neck through SFPN. This design ensures the accurate positioning and detection of small objects, effectively addressing the challenges in complex environments.
• Figure 3: The detailed structure of the MDDC module. The MDDC module initially performs multi-scale decomposition on the input feature map to construct the base representation. Subsequently, through DEIE, it achieves complementary feature fusion in both the spatial and frequency domains. In the frequency domain branch of DEIE, features undergo spectral decomposition, removing low-frequency noise disturbances while enhancing high-frequency feature representations.
• Figure 4: The detailed structure of the WPM. WPM employs a parallel structure consisting of a large kernel convolution, a small kernel convolution, and two strip convolutions.
• Figure 5: The qualitative results of the ablation experiment for all fine-grained categories on the validation set of the VisDrone dataset. Deeper colors indicate higher AP values and better detection performance. "+All" indicates the addition of all components.