Preserving Relative Localization of FoV-Limited Drone Swarm via Active Mutual Observation

TL;DR

This work tackles drift in vision-based relative localization for FoV-limited drone swarms by introducing an active localization correction system that combines a Kalman Filter with a yaw planner to actively induce mutual observations. The approach uses covariance-driven observer–target selection and yaw calculations to ensure mutual detection while maintaining environment awareness, guarded by safety and visibility checks. Results from simulations and real-world indoor/outdoor experiments show substantial drift reduction (up to about 65%) and robust formation maintenance in GPS-denied scenarios, validating scalability to larger swarms with minimal sensor payload. The method offers a practical pathway to reliable formation control for micro aerial vehicles using only a stereo camera and IMU, with demonstrated real-time performance and resilience to measurement noise.

Abstract

Relative state estimation is crucial for vision-based swarms to estimate and compensate for the unavoidable drift of visual odometry. For autonomous drones equipped with the most compact sensor setting -- a stereo camera that provides a limited field of view (FoV), the demand for mutual observation for relative state estimation conflicts with the demand for environment observation. To balance the two demands for FoV limited swarms by acquiring mutual observations with a safety guarantee, this paper proposes an active localization correction system, which plans camera orientations via a yaw planner during the flight. The yaw planner manages the contradiction by calculating suitable timing and yaw angle commands based on the evaluation of localization uncertainty estimated by the Kalman Filter. Simulation validates the scalability of our algorithm. In real-world experiments, we reduce positioning drift by up to 65% and managed to maintain a given formation in both indoor and outdoor GPS-denied flight, from which the accuracy, efficiency, and robustness of the proposed system are verified.

Figures (10)

• Figure 1: Leverage active mutual observation for localization correction in the field experiment. (A) The drones are disorganized due to the drift of the VIO. (B) The mutual observation tasks are assigned, drone 0 observes drone 1, and drone 1 observes drone 2 (white arrows). The yaw rotation (black arrows) can be seen more clearly in the close-up views. (C) After mutual observation, relative localization is corrected. Drones fly in the predefined line formation. (D) Drones conduct environment observation and deform the formation to avoid the tree obstacles. The red curves are the approximate flight paths.
• Figure 2: The contradiction between mutual observation and environment observation. If the blue drone chooses environment observation ($\psi_1$), the mutual measurement cannot be obtained due to the black drone out of the FoV of the black drone. If the blue drone chooses mutual observation ($\psi_2$), the drone may collide with the ellipse-shaped obstacle that has not been seen.
• Figure 3: The system architecture of the active localization correction framework. The notation can be referred to the nomenclature in Sec. \ref{['notation']}. The system consisting of $n$ drones has one leader with a yaw planner and a Kalman Filter deployed. The desired yaw calculated by the leader $\psi_{des}$, the localization result $\mathbf{\hat{X}}$, and mutual observation measurements $\mathbf{z}$ of every drone are transmitted over the wireless network. Based on the covariance $\mathbf{P}$ and the corrected localization $\mathbf{\hat{X}}$ from the Kalman Filter, the yaw planner selects a target and an observer for mutual observation, then calculates the expected yaw $\psi_{des}$, and performs validity checks. After passing the checks, the expected yaw is sent to the drone to execute mutual observation. The drone captured during mutual observation is identified by comparing the estimated position of other drones with the observation measurement. Then the observation measurement is sent to the Kalman Filter for drift estimation. Finally, the estimated drift is used to correct localization information from the VIO.
• Figure 4: A schematic description of the notation. Frame {$G$} means the global frame. Drone $i$ is selected as the observer, and drone $j$ is the target. The estimated global frames of the two drones are not aligned due to the drift $\mathbf{\widetilde{x}}_i$ and $\mathbf{\widetilde{x}}_j$. With the estimated position of itself $\mathbf{\hat{x}}_i$, and the mutual observation measurement $\mathbf{z}_{ij}$, drone $i$ calculates the true position of drone $j$ in the estimated global frame of drone $i$. Then the difference between the true position and the estimated position of drone $j$, $\mathbf{\widetilde{z}}_{ij}$, is the difference between the estimated global frame of two drones.
• Figure 5: An illustration for yaw calculation. The orientation of the observer is aligned with its velocity direction at first. Because of the drift, The true position of the target is different from the position estimated based on the received position. To improve the possibility of successful observation, the observer should turn right until the right boundary of the FoV tangentially intersects the ellipse