Ensuring UAV Safety: A Vision-only and Real-time Framework for Collision Avoidance Through Object Detection, Tracking, and Distance Estimation

TL;DR

This work addresses safe autonomous operation of UAVs by enabling vision-only sense-and-avoid using a monocular camera. It introduces a multi-model pipeline that jointly performs object detection, tracking with camera-motion–compensated Kalman filtering, and lightweight encoder-decoder depth estimation to derive per-pixel distance maps for non-cooperative aerial vehicles. Ground-truth depth is constructed from GPS distances in the AOT dataset, and depth learning is guided by a multi-loss function, evaluated in both regression and classification formulations with sliding-window depth aggregation. The approach achieves promising real-time performance on the AOT benchmark and underscores the importance of depth perception for autonomous collision avoidance, while outlining future work on domain-shift robustness and real-world testing.

Abstract

In the last twenty years, unmanned aerial vehicles (UAVs) have garnered growing interest due to their expanding applications in both military and civilian domains. Detecting non-cooperative aerial vehicles with efficiency and estimating collisions accurately are pivotal for achieving fully autonomous aircraft and facilitating Advanced Air Mobility (AAM). This paper presents a deep-learning framework that utilizes optical sensors for the detection, tracking, and distance estimation of non-cooperative aerial vehicles. In implementing this comprehensive sensing framework, the availability of depth information is essential for enabling autonomous aerial vehicles to perceive and navigate around obstacles. In this work, we propose a method for estimating the distance information of a detected aerial object in real time using only the input of a monocular camera. In order to train our deep learning components for the object detection, tracking and depth estimation tasks we utilize the Amazon Airborne Object Tracking (AOT) Dataset. In contrast to previous approaches that integrate the depth estimation module into the object detector, our method formulates the problem as image-to-image translation. We employ a separate lightweight encoder-decoder network for efficient and robust depth estimation. In a nutshell, the object detection module identifies and localizes obstacles, conveying this information to both the tracking module for monitoring obstacle movement and the depth estimation module for calculating distances. Our approach is evaluated on the Airborne Object Tracking (AOT) dataset which is the largest (to the best of our knowledge) air-to-air airborne object dataset.

Figures (5)

• Figure 1: Our proposed system's pipeline
• Figure 2: Collision avoidance/safe separation thresholds
• Figure 3: Pipeline of the proposed depth estimation model
• Figure 4: Depth estimation visualization for L1, Berhu and multi loss respectively
• Figure 5: Depth estimation ground truth and prediction mask for each classification bin