DREB-Net: Dual-stream Restoration Embedding Blur-feature Fusion Network for High-mobility UAV Object Detection

TL;DR

DREB-Net introduces fast Fourier transform in the early stages of feature extraction, via a learnable frequency domain amplitude modulation module (LFAMM), to adjust feature amplitude and enhance feature processing capability.

Abstract

Object detection algorithms are pivotal components of unmanned aerial vehicle (UAV) imaging systems, extensively employed in complex fields. However, images captured by high-mobility UAVs often suffer from motion blur cases, which significantly impedes the performance of advanced object detection algorithms. To address these challenges, we propose an innovative object detection algorithm specifically designed for blurry images, named DREB-Net (Dual-stream Restoration Embedding Blur-feature Fusion Network). First, DREB-Net addresses the particularities of blurry image object detection problem by incorporating a Blurry image Restoration Auxiliary Branch (BRAB) during the training phase. Second, it fuses the extracted shallow features via Multi-level Attention-Guided Feature Fusion (MAGFF) module, to extract richer features. Here, the MAGFF module comprises local attention modules and global attention modules, which assign different weights to the branches. Then, during the inference phase, the deep feature extraction of the BRAB can be removed to reduce computational complexity and improve detection speed. In loss function, a combined loss of MSE and SSIM is added to the BRAB to restore blurry images. Finally, DREB-Net introduces Fast Fourier Transform in the early stages of feature extraction, via a Learnable Frequency domain Amplitude Modulation Module (LFAMM), to adjust feature amplitude and enhance feature processing capability. Experimental results indicate that DREB-Net can still effectively perform object detection tasks under motion blur in captured images, showcasing excellent performance and broad application prospects. Our source code will be available at https://github.com/EEIC-Lab/DREB-Net.git.

Figures (13)

• Figure 1: Schematic diagram of the task solved in this paper. Existing object detection algorithms often perform poorly on blurry drone images. Through special design, the object detection algorithm can perform well on high-mobility UAVs.
• Figure 2: Overall architecture of the proposed DREB-Net. There are two main branches. One is the object detection main branch, and the other is the Blurry image Restoration Auxiliary Branch(BRAB). The BRAB consists of an encoder-decoder similar to U-Net. The object detection main branch obtains the feature maps of HM (HeatMap), WH (Width and Height) and Reg (Regression) after multiple convolutions and downsampling. In DREB-NET, we have added our innovative modules, MAGFF and LFAMM, which will be introduced in detail later.
• Figure 3: MGAFF module. It consists of local attention submodules and global attention submodules, which process input features through different paths and select and fuse features through the sigmoid function. It is used to optimize the integration of global information and improve the detail perception ability of features when the object detection main branch and the BRAB perform feature fusion.
• Figure 4: LFAMM module. It relies on frequency domain processing to enhance the feature expression ability of input images. It uses the Real Fast Fourier Transform (RFFT) and its inverse (IRFFT) and convolution operation to preserve the spatial information of the image while adjusting its frequency components to enhance the details and quality of the image.
• Figure 5: Statistics histogram of image count by PSNR and average SSIM per PSNR for the blurred image datasets. (a) on the blurred VisDrone-2019-DET dataset. (b) on the blurred UAVDT dataset.