UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehicles

TL;DR

UAV3D, a benchmark designed to advance research in both 3D and collaborative 3D perception tasks with UAVs, is introduced, providing the benchmark for four 3D perception tasks: single-UAV 3D object detection, single-UAV object tracking, collaborative-UAV 3D object detection, and collaborative-UAV object tracking.

Abstract

Unmanned Aerial Vehicles (UAVs), equipped with cameras, are employed in numerous applications, including aerial photography, surveillance, and agriculture. In these applications, robust object detection and tracking are essential for the effective deployment of UAVs. However, existing benchmarks for UAV applications are mainly designed for traditional 2D perception tasks, restricting the development of real-world applications that require a 3D understanding of the environment. Furthermore, despite recent advancements in single-UAV perception, limited views of a single UAV platform significantly constrain its perception capabilities over long distances or in occluded areas. To address these challenges, we introduce UAV3D, a benchmark designed to advance research in both 3D and collaborative 3D perception tasks with UAVs. UAV3D comprises 1,000 scenes, each of which has 20 frames with fully annotated 3D bounding boxes on vehicles. We provide the benchmark for four 3D perception tasks: single-UAV 3D object detection, single-UAV object tracking, collaborative-UAV 3D object detection, and collaborative-UAV object tracking. Our dataset and code are available at https://huiyegit.github.io/UAV3D_Benchmark/.

Figures (6)

• Figure 1: An example from the UAV3D dataset. From the top to bottom, they are 5 different RGB images from different camera views, images with 3D bounding boxes, and images with semantic labels.
• Figure 2: Sensor setup for our data collection platform.
• Figure 3: UAV swarm with a cross-shaped formation.
• Figure 4: Visualization of the back view for collaborative-UAV 3D object detection results on UAV3D. Red boxes represent the predictions, while blue boxes indicate the ground truth.
• Figure 5: Number of annotations per category in UAV3D training, validation and test set. The vehicle categories are almost evenly distributed.