FlyPose: Towards Robust Human Pose Estimation From Aerial Views

TL;DR

This work tackles robust human pose estimation from aerial viewpoints by delivering FlyPose, a lightweight top-down pipeline designed for onboard UAV deployment. The system combines a compact detector (RT-DETRv2-S) with a heatmap-based 2D pose estimator (ViTPose-S) and is trained via multi-dataset augmentation to handle small, top-down, and occluded subjects in real time. Key contributions include significant detection and pose estimation gains across multiple aerial datasets, the FlyPose-104 challenging aerial pose benchmark, and a real-flight onboard deployment achieving around 20 ms per frame, enabling approximately 25 frames per second for responsive downstream tasks. The practical impact lies in enabling safe, gesture-informed human-robot interactions and real-time perception for UAV applications in populated environments, with a flexible, edge-friendly architecture suitable for resource-constrained platforms.

Abstract

Unmanned Aerial Vehicles (UAVs) are increasingly deployed in close proximity to humans for applications such as parcel delivery, traffic monitoring, disaster response and infrastructure inspections. Ensuring safe and reliable operation in these human-populated environments demands accurate perception of human poses and actions from an aerial viewpoint. This perspective challenges existing methods with low resolution, steep viewing angles and (self-)occlusion, especially if the application demands realtime feasibile models. We train and deploy FlyPose, a lightweight top-down human pose estimation pipeline for aerial imagery. Through multi-dataset training, we achieve an average improvement of 6.8 mAP in person detection across the test-sets of Manipal-UAV, VisDrone, HIT-UAV as well as our custom dataset. For 2D human pose estimation we report an improvement of 16.3 mAP on the challenging UAV-Human dataset. FlyPose runs with an inference latency of ~20 milliseconds including preprocessing on a Jetson Orin AGX Developer Kit and is deployed onboard a quadrotor UAV during flight experiments. We also publish FlyPose-104, a small but challenging aerial human pose estimation dataset, that includes manual annotations from difficult aerial perspectives: https://github.com/farooqhassaan/FlyPose.

Figures (9)

• Figure 1: Applications like urban traffic monitoring and sling-load cargo deliveries with unmanned aerial vehicles pose a major challenge for human pose estimation due to varying person sizes, occlusion and low-resolution overhead imagery brenner2023udwdu2019visdroneyang2019top.
• Figure 2: Two examples from our FlyPose-104 dataset with manually annotated bounding boxes and poses, featuring frequent self-occlusions of lower body and facial joints (marked in red) yang2019topbrenner2023udw.
• Figure 3: System overview: the bottom illustrates the FlyPose pipeline, where the detector and pose estimation model are trained separately, the top is an example for how the aggregated information can be used for downstream tasks within the drone system for various applications.
• Figure 4: Pose Estimation performance of pretrained models on the UAV-Human dataset, plotted against their latency on an RTX A6000 GPU. Each circle's radius is proportional to the model parameter count.
• Figure 5: Qualitative detections (red) on the VisDrone2019-DET (top), FlyPose-104 (bottom left) and HIT-UAV (bottom right).