Bio-inspired visual relative localization for large swarms of UAVs

TL;DR

The paper tackles scalable relative localization for large UAV swarms using monocular vision by regressing neighbor density over distance, rather than estimating individual neighbor poses. A CNN-based density estimator outputs per-cell distributions across distance bins $d$ with ground-truth smoothed by a Gaussian, trained via a weighted Euclidean loss to emphasize near-field targets. The approach demonstrates superior robustness and efficiency compared to detector-based methods, validated on a large synthetic dataset and extended to high-density scenarios and real-world data with successful generalization after fine-tuning. This density-regression framework enables faster onboard processing and better scalability, making it suitable as the primary sensing modality for swarm stabilization and collision avoidance in infrastructure-light deployments.

Abstract

We propose a new approach to visual perception for relative localization of agents within large-scale swarms of UAVs. Inspired by biological perception utilized by schools of sardines, swarms of bees, and other large groups of animals capable of moving in a decentralized yet coherent manner, our method does not rely on detecting individual neighbors by each agent and estimating their relative position, but rather we propose to regress a neighbor density over distance. This allows for a more accurate distance estimation as well as better scalability with respect to the number of neighbors. Additionally, a novel swarm control algorithm is proposed to make it compatible with the new relative localization method. We provide a thorough evaluation of the presented methods and demonstrate that the regressing approach to distance estimation is more robust to varying relative pose of the targets and that it is suitable to be used as the main source of relative localization for swarm stabilization.

Figures (9)

• Figure 1: Top: An illustration of the Gaussian smoothing and the CNN prediction of the distribution of UAV in the image over distance. The input image is shown on the right. Bottom: The output of the CNN with $w_{\text{out}} = h_{\text{out}} = 3$ for the input image. The predicted density for a given grid cell and distance is marked with a circle of the corresponding size and color from blue to yellow.
• Figure 2: Visualization of the network architecture for $w_{\text{in}} = h_{\text{in}} = 300$, $w_{\text{out}} = h_{\text{out}} = 3$, and $n_{\text{bin}} = 50$.
• Figure 3: Comparison of synthetic (a) and real (b) images of the MRS F450 platform MRS:platform used in this work.
• Figure 4: Distribution of images in the dataset based on the total number of UAV in the given image. The exact counts are displayed above the bars.
• Figure 5: Distribution of UAV in the dataset images based on their corresponding distance bin.