Demonstrating ViSafe: Vision-enabled Safety for High-speed Detect and Avoid

TL;DR

ViSafe delivers a vision-only Detect and Avoid solution for high-speed, SWaP-C constrained aerial platforms by integrating a learning-based edge perception pipeline with multi-camera fusion and a CBF-based safety layer implemented as a real-time QP. The approach is validated across hardware experiments, hardware-in-the-loop digital twins, and real-world flights, achieving provable self-separation at closure rates up to 144 km/h and showing robustness to weather and lighting variations. A digital twin is used to benchmark and bound real-world performance, while the hardware prototype demonstrates on-board processing, low-latency perception, and safe maneuver execution. Overall, ViSafe establishes a practical, high-speed, vision-only DAA framework with provable safety guarantees that can operate within SWaP-C constraints for autonomous aerial navigation.

Abstract

Assured safe-separation is essential for achieving seamless high-density operation of airborne vehicles in a shared airspace. To equip resource-constrained aerial systems with this safety-critical capability, we present ViSafe, a high-speed vision-only airborne collision avoidance system. ViSafe offers a full-stack solution to the Detect and Avoid (DAA) problem by tightly integrating a learning-based edge-AI framework with a custom multi-camera hardware prototype designed under SWaP-C constraints. By leveraging perceptual input-focused control barrier functions (CBF) to design, encode, and enforce safety thresholds, ViSafe can provide provably safe runtime guarantees for self-separation in high-speed aerial operations. We evaluate ViSafe's performance through an extensive test campaign involving both simulated digital twins and real-world flight scenarios. By independently varying agent types, closure rates, interaction geometries, and environmental conditions (e.g., weather and lighting), we demonstrate that ViSafe consistently ensures self-separation across diverse scenarios. In first-of-its-kind real-world high-speed collision avoidance tests with closure rates reaching 144 km/h, ViSafe sets a new benchmark for vision-only autonomous collision avoidance, establishing a new standard for safety in high-speed aerial navigation.

Figures (6)

• Figure 1: We demonstrate ViSafe, a high-speed vision-only airborne collision avoidance system.Top: Rendering of a real-world flight test log where the ViSafe system detects an incoming intruder with a 144 km/h closure rate and performs an avoidance maneuver to ensure safe separation. The annotations showcase the detections and multi-camera tracks from the vision-based aircraft detection and tracking system, while the trajectory shows the log of the performed real-world avoidance maneuver. The numbered annotations showcase the different stages of the flight test, from intruder detection to avoidance completion. Bottom: ViSafe payload, ownship, and intruder platforms used in the real-world flight tests.
• Figure 2: Overview of our ViSafe framework for real-world testing and hardware-in-the-loop simulation: Firstly, the onboard sensors or digital twin simulation stream the multi-cam videos to the AirTrack visual detection module, which detects the intruder across multiple views. Then, these detections, along with the ownship state information, are sent to the multi-view fusion and coordinate frame conversion module, which then tracks the intruder and sends the intruder state information along with the ownship state information to the CBF. The CBF uses the nominal global plan and the safety violation assessment to compute modifications to the nominal control input in case of violation. This safe control output is then sent to the drone autopilot system for execution. This loop continues to operate in real-time until the flight test is complete.
• Figure 3: Encounter geometry and information required for vision-enabled collision avoidance. The velocity of the ownship $v_{own}$ and heading with respect to North $\chi_{own}$ are obtained from the ownship odometry. The intruder's range $d$, azimuth $\theta$, velocity $v_{int}$, and heading with respect to North $\chi_{int}$ are obtained using the visual detection module.
• Figure 4: Diversity of the airborne collision testing scenarios: (a) The various encounter geometries used for real-world & simulation flight testing. (b) The diverse weather and lighting conditions that were used to evaluate ViSafe's robustness in simulation. In the simulation, based on the chosen encounter geometry, the intruder position is sampled randomly on the flying corridor.
• Figure 5: Average horizontal rate of closure comparisons across different encounter geometries in real-world testing: Higher values indicate that agents are moving apart, showcasing diverging & safe trajectories. Under different testing scenarios, it can be seen that ViSafe consistently shows a significant boost in average HROC over the nominal plan.