U2UData: A Large-scale Cooperative Perception Dataset for Swarm UAVs Autonomous Flight

TL;DR

U2UData addresses the limited generalization of single-UAV perception in cluttered and uncertain environments by introducing a large-scale, real2sim cooperative dataset for swarm UAVs, collected in the U2USim environment across diverse terrains and weather. The authors propose two tasks—cooperative 3D object detection and cooperative 3D object tracking—and benchmark multiple state-of-the-art cooperative perception methods, notably highlighting the benefits and bandwidth costs of intermediate fusion techniques. The dataset comprises 315K LiDAR frames, 945K RGB/depth frames, 2.41M 3D bounding boxes for three animal classes, and multimodal weather-related signals, enabling robust cross-UAV collaboration. U2USim provides a realistic, weather-informed simulator map based on Yunnan data to bridge the sim-to-real gap, and results demonstrate substantial performance gains under cooperative schemes, while exposing challenges in asynchronous communication. The work provides public release of data, benchmarks, and models to accelerate swarm-UAV perception research.

Abstract

Modern perception systems for autonomous flight are sensitive to occlusion and have limited long-range capability, which is a key bottleneck in improving low-altitude economic task performance. Recent research has shown that the UAV-to-UAV (U2U) cooperative perception system has great potential to revolutionize the autonomous flight industry. However, the lack of a large-scale dataset is hindering progress in this area. This paper presents U2UData, the first large-scale cooperative perception dataset for swarm UAVs autonomous flight. The dataset was collected by three UAVs flying autonomously in the U2USim, covering a 9 km$^2$ flight area. It comprises 315K LiDAR frames, 945K RGB and depth frames, and 2.41M annotated 3D bounding boxes for 3 classes. It also includes brightness, temperature, humidity, smoke, and airflow values covering all flight routes. U2USim is the first real-world mapping swarm UAVs simulation environment. It takes Yunnan Province as the prototype and includes 4 terrains, 7 weather conditions, and 8 sensor types. U2UData introduces two perception tasks: cooperative 3D object detection and cooperative 3D object tracking. This paper provides comprehensive benchmarks of recent cooperative perception algorithms on these tasks.

Figures (3)

• Figure 1: U2USim overview. The upper left is U2USim's topographic map, the upper middle is U2USim's vegetation conditions, and the upper right is U2USim's tasks. Below are 6 weather views, from left to right: sunny, rain, snow, fog, sandstorm, and thunder. Since wind is invisible, there is no collection view.
• Figure 2: Sensor setup for our data collection platform.
• Figure 3: Three types of swarm-UAVs formation. Left: Discipline formation mode, where swarm-UAVs keep a consistent and relatively static array; Medium: Fixed formation mode, where each UAV navigates independently with a fixed path; Right: Autonomous formation mode, where each UAV flies autonomously.