Person Recognition in Aerial Surveillance: A Decade Survey

TL;DR

The paper surveys over 150 papers on human-centric aerial surveillance, dissecting detection, identification, and re-identification tasks from drone-mounted sensors. It provides a structured framework examining aerial-specific challenges, public datasets, and state-of-the-art approaches, highlighting domain shifts and modality gaps relative to ground-based data. Key contributions include a taxonomy of challenges, a dataset landscape analysis, and a meta-analysis of methods across tasks, with open questions on data, robustness, and privacy. The work emphasizes the need for aerial-aware architectures, multimodal fusion, and responsible deployment to realize practical and trustworthy aerial surveillance systems.

Abstract

The rapid emergence of airborne platforms and imaging sensors is enabling new forms of aerial surveillance due to their unprecedented advantages in scale, mobility, deployment, and covert observation capabilities. This paper provides a comprehensive overview of 150+ papers over the last 10 years of human-centric aerial surveillance tasks from a computer vision and machine learning perspective. It aims to provide readers with an in-depth systematic review and technical analysis of the current state of aerial surveillance tasks using drones, UAVs, and other airborne platforms. The object of interest is humans, where human subjects are to be detected, identified, and re-identified. More specifically, for each of these tasks, we first identify unique challenges in performing these tasks in an aerial setting compared to the popular ground-based setting and subsequently compile and analyze aerial datasets publicly available for each task. Most importantly, we delve deep into the approaches in the aerial surveillance literature with a focus on investigating how they presently address aerial challenges and techniques for improvement. We conclude the paper by discussing the gaps and open research questions to inform future research avenues.

Figures (10)

• Figure 1: The rise of aerial surveillance. (1) The first column shows the eyes in the sky Gorgon Stare EyesInTheSky deployed in Iraq, using an MQ-9 drone (costs $$17M$) flying at an altitude of $7,000$ m, able to capture simultaneously an area of $10 \times 10$ km, with a $1.8$ GB sensor at a Ground Sampling Distance (GSD) of $15$ cm. (2) The second column is the off-the-shelf DJI Mavic 2 (costs $$3K$) flying at an altitude of $100$ m. One example public dataset collected with DJI Mavic 2 was VisDrones VisDrones. (3) The third column is Supervolo - a gasoline aircraft that can fly 8 hours at an altitude of $400$ m, able to capture an area of $1 \times 1$ km simultaneously, at a GSD of 20 cm. One example public dataset collected with Supervolo was BRIAR BRIARdataset.
• Figure 2: Increasing trend of publications in human-centric aerial surveillance over the last ten years. (Data from Scopus advanced search: (TITLE (aerial OR uav OR drone) AND TITLE (detect* OR identi* OR recogni* OR re-id) AND TITLE (human OR person OR pedestrian OR object) AND NOT TITLE (vehicle)) AND PUBYEAR > 2014.)
• Figure 3: Three challenge dimensions of surveillance—subject-camera distance, altitude, and camera motion - highlighting the contrast between aerial and ground-based systems.
• Figure 4: Challenges for aerial object detection: (i) low resolution, (ii) a wide range of scales, (iii) arbitrary viewing angles, (iv) non-uniformly distributed, (v) illumination. Images from the TinyPersons TinyPersons and VisDrone VisDrones datasets.
• Figure 5: State-of-the-art generic detectors drop the performance when shifting to the aerial data. The figure compares the detection accuracy (AP50) of Cascade R-CNN CascadeRCNN and Faster R-CNN FasterRCNN on two ground-based (COCO and VOC) and two aerial (VisDrone and TinyPersons) datasets.