DDOS: The Drone Depth and Obstacle Segmentation Dataset

TL;DR

DDOS tackles the lack of aerial-scale training data for autonomous drones by introducing a large-scale synthetic dataset with depth maps, pixel-wise semantic segmentation for ten classes, optical flow, and surface normals, generated in two weather-varied environments via AirSim. A key contribution is the introduction of drone-specific depth metrics, including class-wise AbsRel, to evaluate depth accuracy for thin structures such as wires, where existing benchmarks fall short. Baseline experiments with BinsFormer, SimIPU, and DepthFormer reveal that Ultra Thin objects remain poorly estimated, underscoring the need for methods tailored to thin-structure depth. Overall, DDOS provides a comprehensive, safety-focused platform to advance obstacle segmentation and depth estimation for robust drone navigation in real-world, challenging conditions.

Abstract

The advancement of autonomous drones, essential for sectors such as remote sensing and emergency services, is hindered by the absence of training datasets that fully capture the environmental challenges present in real-world scenarios, particularly operations in non-optimal weather conditions and the detection of thin structures like wires. We present the Drone Depth and Obstacle Segmentation (DDOS) dataset to fill this critical gap with a collection of synthetic aerial images, created to provide comprehensive training samples for semantic segmentation and depth estimation. Specifically designed to enhance the identification of thin structures, DDOS allows drones to navigate a wide range of weather conditions, significantly elevating drone training and operational safety. Additionally, this work introduces innovative drone-specific metrics aimed at refining the evaluation of algorithms in depth estimation, with a focus on thin structure detection. These contributions not only pave the way for substantial improvements in autonomous drone technology but also set a new benchmark for future research, opening avenues for further advancements in drone navigation and safety.

Figures (9)

• Figure 1: Examples from our DDOS dataset. This figure showcases an overview of the DDOS dataset's multifaceted annotations. It includes RGB images from drone flights, depth maps (0.0--100m), pixel-wise semantic segmentation, optical flow and surface normals, illustrating the dataset's richness and diversity.
• Figure 2: Distribution of class labels within DDOS. DDOS effectively captures the presence of various thin object classes, which are characterized by a relatively sparse distribution of pixels within each image. Despite their limited pixel coverage, these thin object classes are well-represented in DDOS, ensuring comprehensive coverage and enabling robust training and evaluation of algorithms specifically designed to address the challenges posed by such objects.
• Figure 3: Distribution of pitch and roll angles. The colors represent the intensity levels, with warmer colors indicating higher occurrences. Flight characteristics vary between each flight, as highlighted by the diverse pitch and roll degrees. The pitch is negative when the drone is accelerating forward and positive when braking or to go backwards. Emergency braking is often accompanied with a sharp turn, either to the left or to the right.
• Figure 4: Illustrated flight paths. The figure presents a collection of 50 randomly selected flight paths conducted within the same environment. The paths exhibit significant variations in trajectory, highlighting the diverse nature of drone flights.
• Figure 5: Overhead view of relative flight paths with a normalized starting point. In this visualization the starting location and direction have been normalized to highlight the various relative shapes of the flight paths. The actual starting locations are randomly initialized, as shown in \ref{['fig:example_flight_paths']}.