Adaptive Data Transport Mechanism for UAV Surveillance Missions in Lossy Environments

TL;DR

This paper adopts a different perspective and offers an alternative AI-driven scheduling policy that prioritizes selecting regions of the image that significantly contribute to the mission objective and develops a Deep Reinforcement Learning (DRL) framework that assigns higher transmission probabilities to patches that present higher overlaps with the detected object of interest.

Abstract

Unmanned Aerial Vehicles (UAVs) play an increasingly critical role in Intelligence, Surveillance, and Reconnaissance (ISR) missions such as border patrolling and criminal detection, thanks to their ability to access remote areas and transmit real-time imagery to processing servers. However, UAVs are highly constrained by payload size, power limits, and communication bandwidth, necessitating the development of highly selective and efficient data transmission strategies. This has driven the development of various compression and optimal transmission technologies for UAVs. Nevertheless, most methods strive to preserve maximal information in transferred video frames, missing the fact that only certain parts of images/video frames might offer meaningful contributions to the ultimate mission objectives in the ISR scenarios involving moving object detection and tracking (OD/OT). This paper adopts a different perspective, and offers an alternative AI-driven scheduling policy that prioritizes selecting regions of the image that significantly contributes to the mission objective. The key idea is tiling the image into small patches and developing a deep reinforcement learning (DRL) framework that assigns higher transmission probabilities to patches that present higher overlaps with the detected object of interest, while penalizing sharp transitions over consecutive frames to promote smooth scheduling shifts. Although we used Yolov-8 object detection and UDP transmission protocols as a benchmark testing scenario the idea is general and applicable to different transmission protocols and OD/OT methods. To further boost the system's performance and avoid OD errors for cluttered image patches, we integrate it with interframe interpolations.

Figures (8)

• Figure 1: Overview of the UAV-based surveillance system with a feedback loop between the UAV and GCS for real-time object detection and monitoring.
• Figure 2: Sample frames and labeled with object annotations from the AU-AIR dataset.
• Figure 3: Visualization of the DQN training results: (a) A heatmap representing the varying importance of different regions within the frame, where lighter cells indicate higher importance or the likelihood of object presence as predicted by the DQN. (b) The truck's path, visualized by blending sequential frames, demonstrates the DQN's ability to accurately predict the movement trajectory based on past observation
• Figure 4: Selected cells for transmission based on the top probabilities using different masks.
• Figure 5: Illustration of the frame reconstruction process: the received frame, containing only selectively transmitted cells, is shown on the left. The right image demonstrates the result after applying interpolation to the missing areas, effectively reconstructing a more coherent and visually complete frame.