NOMAD: A Natural, Occluded, Multi-scale Aerial Dataset, for Emergency Response Scenarios

TL;DR

NOMAD tackles the critical challenge of detecting humans in occluded aerial views for emergency response by providing a richly annotated, multimodal benchmark. The dataset captures natural, occluded, and multi-scale scenarios across 100 actors at five distances, totaling 42,825 frames with 10 levels of visibility and extensive metadata. Baseline evaluations using state-of-the-art detectors reveal strong performance at close, unobstructed views but substantial degradation as occlusion and distance increase, underscoring the need for occlusion-aware models and temporal information. The work offers a practical, real-world resource for advancing aerial search and rescue performance and guiding future occlusion-robust perception systems for sUAS.

Abstract

With the increasing reliance on small Unmanned Aerial Systems (sUAS) for Emergency Response Scenarios, such as Search and Rescue, the integration of computer vision capabilities has become a key factor in mission success. Nevertheless, computer vision performance for detecting humans severely degrades when shifting from ground to aerial views. Several aerial datasets have been created to mitigate this problem, however, none of them has specifically addressed the issue of occlusion, a critical component in Emergency Response Scenarios. Natural, Occluded, Multi-scale Aerial Dataset (NOMAD) presents a benchmark for human detection under occluded aerial views, with five different aerial distances and rich imagery variance. NOMAD is composed of 100 different Actors, all performing sequences of walking, laying and hiding. It includes 42,825 frames, extracted from 5.4k resolution videos, and manually annotated with a bounding box and a label describing 10 different visibility levels, categorized according to the percentage of the human body visible inside the bounding box. This allows computer vision models to be evaluated on their detection performance across different ranges of occlusion. NOMAD is designed to improve the effectiveness of aerial search and rescue and to enhance collaboration between sUAS and humans, by providing a new benchmark dataset for human detection under occluded aerial views. Full dataset can be found at: https://github.com/ArtRuss/NOMAD.

Figures (9)

• Figure 1: Development and characteristics of NOMAD. Integration of sUAS into emergency response scenarios have aided first responders and rescued victims news_drowningnews_hikingAfricanews_lostWoodsnews_earthquakenews_girlnews_firenews_teamworknews_oldadultnews_firehomes (first column). Nevertheless, multiple challenges inherent to these situations degrade CV performance and halt sUAS full integration, including the highly prevalent problem of occlusion (second column). We present NOMAD, Natural Occluded Multi-scale Aerial Dataset, providing the research community with emergency response related videos and selected frames, as well as rich metadata and annotations, including a visibility label (third column). Facing emergency response scenarios, key characteristics of our dataset are: Natural: diversity of filming locations, cross-seasonal imagery, including winter scenarios, and a demographic diversity on gender, age and race, ranging from 18 to 78 years old, and including White Caucasians, Latinos, African descent, Asians, South Asians, Middle Eastern and Pacific Islander; Occluded: 10 defined ranges of occlusion, with a visibility label assigned to every bounding box; Multi-scale: five different distances, ranging from 10m to 90m altitude, and a ground reference view for every actor.
• Figure 2: Filming process. Sample positioning of the sUAS at 10m horizontally and vertically from the actor's starting location.
• Figure 3: Keyframe selection process. (a) Sample routine with 12 keyframes selected. Sampling for occluded views is indicated by red-dashed arrows. (b) Sample Activity labels for the keyframes illustrated. Hiding (L) represents Hiding (Laying).
• Figure 4: Visibility label calculation. Percentages assigned to each body part of a person.
• Figure 5: Distribution of the filming locations for the 100 actors.