Coverage-Recon: Coordinated Multi-Drone Image Sampling with Online Map Feedback

TL;DR

Coverage-Recon addresses collaborative 3D map reconstruction by coupling an extended angle-aware coverage controller with online map feedback from real-time NeuralRecon meshes. The framework maintains a five-dimensional drone control input, enforces safety via Control Barrier Functions, and introduces mesh-change driven updates to the importance indices guiding sampling. Online feedback is incorporated through a jump in point importance $\phi_j$ triggered by mesh evolution signals $h_2$, with two mesh-change quantification methods (3D Grid and M3C2) to balance fidelity and computation. Simulations and indoor experiments show that online map feedback yields more complete and accurate reconstructions than baselines, with scalable performance as the number of drones increases.

Abstract

This article addresses collaborative 3D map reconstruction using multiple drones. Achieving high-quality reconstruction requires capturing images of keypoints within the target scene from diverse viewing angles, and coverage control offers an effective framework to meet this requirement. Meanwhile, recent advances in real-time 3D reconstruction algorithms make it possible to render an evolving map during flight, enabling immediate feedback to guide drone motion. Building on this, we present Coverage-Recon, a novel coordinated image sampling algorithm that integrates online map feedback to improve reconstruction quality on-the-fly. In Coverage-Recon, the coordinated motion of drones is governed by a Quadratic Programming (QP)-based angle-aware coverage controller, which ensures multi-viewpoint image capture while enforcing safety constraints. The captured images are processed in real time by the NeuralRecon algorithm to generate an evolving 3D mesh. Mesh changes across the scene are interpreted as indicators of reconstruction uncertainty and serve as feedback to update the importance index of the coverage control as the map evolves. The effectiveness of Coverage-Recon is validated through simulation and experiments, demonstrating both qualitatively and quantitatively that incorporating online map feedback yields more complete and accurate 3D reconstructions than conventional methods. Project page: https://htnk-lab.github.io/coverage-recon/

Figures (14)

• Figure 1: Coverage-Recon enables autonomous multi-drone reconstruction through a closed feedback loop between image sampling and mapping. Left: drones use angle-aware coverage control with a QP-based controller to capture diverse viewpoints. Right: NeuralRecon sun2021neuralrecon generates an evolving 3D mesh that guides drones to focus on under-reconstructed regions.
• Figure 2: Illustration of the geometric relationship between drones and observation points. Drones in flight region $\mathcal{P}$ observe target region $\mathcal{B}$, adjusting camera yaw $\theta^h_i$ and pitch $\theta^v_i$ to capture diverse viewpoints for 3D reconstruction.
• Figure 3: Visualization of the sensing performance metrics for a drone at state $p_i$ observing an observation point $q_j$, showing the distance $D$, deviation angles $\varphi_1$ and $\varphi_2$, and their associated direction vectors.
• Figure 4: The NeuralRecon pipeline sun2021neuralrecon takes posed monocular RGB keyframes ${(I_t,\xi_t)}$ as input and outputs a triangular mesh $(V^t,F^t)$, updated at discrete events $t \in \mathcal{T}$.
• Figure 5: System architecture of Coverage-Recon. The left block shows the controller framework implemented in ROS2, which is used in both simulation and real-world experiments. In simulation, the real environment is replaced by a virtual 3D scene created in Unity (right block), which serves as the target to be reconstructed by the drone.

Theorems & Definitions (6)

• Remark 1
• Theorem 4.1
• proof
• Remark 2
• Theorem 4.2
• proof