Fusion Flow-enhanced Graph Pooling Residual Networks for Unmanned Aerial Vehicles Surveillance in Day and Night Dual Visions

TL;DR

The paper tackles UAV detection in no-fly zones under day and night conditions using dual RGB-IR vision. It introduces OF-GPRN, a framework that fuses RGB and IR frames, extracts motion with optical flow, and leverages a Graph Residual Split Attention Network with a Graph-Pooling Feature Pyramid to capture multi-scale, graph-structured features. The approach achieves a peak mAP of 87.8% on the UAVCatch dataset, significantly surpassing ResGCN baselines, with robust training stability and strong ablations validating the components. This work advances practical UAV surveillance by combining optical-flow-driven fusion, graph-based attention, and multi-scale feature mapping to handle challenging illumination and background-similarity scenarios.

Abstract

Recognizing unauthorized Unmanned Aerial Vehicles (UAVs) within designated no-fly zones throughout the day and night is of paramount importance, where the unauthorized UAVs pose a substantial threat to both civil and military aviation safety. However, recognizing UAVs day and night with dual-vision cameras is nontrivial, since red-green-blue (RGB) images suffer from a low detection rate under an insufficient light condition, such as on cloudy or stormy days, while black-and-white infrared (IR) images struggle to capture UAVs that overlap with the background at night. In this paper, we propose a new optical flow-assisted graph-pooling residual network (OF-GPRN), which significantly enhances the UAV detection rate in day and night dual visions. The proposed OF-GPRN develops a new optical fusion to remove superfluous backgrounds, which improves RGB/IR imaging clarity. Furthermore, OF-GPRN extends optical fusion by incorporating a graph residual split attention network and a feature pyramid, which refines the perception of UAVs, leading to a higher success rate in UAV detection. A comprehensive performance evaluation is conducted using a benchmark UAV catch dataset. The results indicate that the proposed OF-GPRN elevates the UAV mean average precision (mAP) detection rate to 87.8%, marking a 17.9% advancement compared to the residual graph neural network (ResGCN)-based approach.

Figures (10)

• Figure 1: Unauthorized UAVs patrol in the no-fly zone, which disturbs the takeoff and landing of the flight or spies on confidential military operations.
• Figure 2: The proposed OF-GPRN system architecture for UAV detection and tracking in day and night vision.
• Figure 3: In deep overview of the split graph attention is shown here which represents each layers of the graph convolutional network.
• Figure 4: The scaling factor's impact on cross-entropy to automatically downweight loose samples during training.
• Figure 5: The columns on the far right are RGB frames with object optical flow. The middle column shows the IR frames and optical flow. In contrast, the most right-hand column combines RGB-IR and then displays the effect of UAV optical flow.