Threat-Aware UAV Dodging of Human-Thrown Projectiles with an RGB-D Camera

TL;DR

This work addresses the threat of sudden human-initiated projectile attacks on UAVs by introducing a real-time, RGB-D perception system that predicts projectile trajectories from human pose. It combines Pose-Aware Projectile Trajectory Prediction (PAPT) with Uncertainty-Aware Dodging (UAD), employing an ivory-shaped uncertainty model and MINCO trajectory optimization to plan safe evasive maneuvers on CPU hardware. The approach achieves long-range detection (6 m), low latency (26.4 ms), and high dodging success (up to 100% in simulations; 96.7% in comparisons with a baseline), demonstrating robustness across lighting, occlusion, and multi-threat scenarios. The framework is modular and transferable across UAV platforms, with potential applications in sports, security, and crowd-safe navigation, and future work toward richer semantic interpretation of human intent.

Abstract

Uncrewed aerial vehicles (UAVs) performing tasks such as transportation and aerial photography are vulnerable to intentional projectile attacks from humans. Dodging such a sudden and fast projectile poses a significant challenge for UAVs, requiring ultra-low latency responses and agile maneuvers. Drawing inspiration from baseball, in which pitchers' body movements are analyzed to predict the ball's trajectory, we propose a novel real-time dodging system that leverages an RGB-D camera. Our approach integrates human pose estimation with depth information to predict the attacker's motion trajectory and the subsequent projectile trajectory. Additionally, we introduce an uncertainty-aware dodging strategy to enable the UAV to dodge incoming projectiles efficiently. Our perception system achieves high prediction accuracy and outperforms the baseline in effective distance and latency. The dodging strategy addresses temporal and spatial uncertainties to ensure UAV safety. Extensive real-world experiments demonstrate the framework's reliable dodging capabilities against sudden attacks and its outstanding robustness across diverse scenarios.

Figures (7)

• Figure 1: Upon detecting the human's intention to attack with a tennis ball, the UAV executes a rapid dodging maneuver to avoid the sudden, fast projectile.
• Figure 2: System overview. The system architecture for UAV dodging of human-initiated projectile attacks consists of two main modules: PAPT and UAD. PAPT takes RGB-D camera streams as input, processes RGB-D camera streams through 2D HPE and depth value processing to track 3D body keypoints associated with projectile throwing, and calculates pre- and post-release trajectory. Using initial projectile trajectory predictions, the UAD employs an ivory-shaped uncertainty model and identifies surviving trajectories and regions for assessing UAV collision risk. It then incorporates penalties from the proximity and relative velocity of surviving projectile trajectories to generate a safe dodging trajectory.
• Figure 3: Ivory-shaped uncertainty model: The prediction error in the projectile's trajectory increases over time. Based on this, we construct an ivory-shaped uncertainty region around the predicted trajectory. A collision risk is deemed present when the UAV’s current position or planned trajectory intersects with this region.
• Figure 4: (a) A participant performs arm motions while wearing a 3D-printed device equipped with mocap markers. (b) Comparison of PAPT-estimated 3D trajectory with ground truth from the mocap. (c) Positional and velocity differences between the estimated and ground truth along the x-, y-, and z-axes.
• Figure 5: Timeline analysis illustrating system performances for attack dodging within the same simulated environment. The figure presents a horizontal time axis with processing operations at distinct time nodes for two systems.