MULAN-WC: Multi-Robot Localization Uncertainty-aware Active NeRF with Wireless Coordination

TL;DR

This paper introduces a method for using wireless Angle-of-Arrival and ranging measurements to estimate relative poses between robots, as well as quantifying and incorporating the uncertainty embedded in the wireless localization of these pose estimates into the NeRF training loss to mitigate the impact of inaccurate camera poses.

Abstract

This paper presents MULAN-WC, a novel multi-robot 3D reconstruction framework that leverages wireless signal-based coordination between robots and Neural Radiance Fields (NeRF). Our approach addresses key challenges in multi-robot 3D reconstruction, including inter-robot pose estimation, localization uncertainty quantification, and active best-next-view selection. We introduce a method for using wireless Angle-of-Arrival (AoA) and ranging measurements to estimate relative poses between robots, as well as quantifying and incorporating the uncertainty embedded in the wireless localization of these pose estimates into the NeRF training loss to mitigate the impact of inaccurate camera poses. Furthermore, we propose an active view selection approach that accounts for robot pose uncertainty when determining the next-best views to improve the 3D reconstruction, enabling faster convergence through intelligent view selection. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of our framework in theory and in practice. Leveraging wireless coordination and localization uncertainty-aware training, MULAN-WC can achieve high-quality 3d reconstruction which is close to applying the ground truth camera poses. Furthermore, the quantification of the information gain from a novel view enables consistent rendering quality improvement with incrementally captured images by commending the robot the novel view position. Our hardware experiments showcase the practicality of deploying MULAN-WC to real robotic systems.

Figures (5)

• Figure 1: Simulation of the AoA variance methodology. (a) Measured AoA profile $\overline{F}_{\alpha\beta}(\phi,\theta)$ from a simulated measured wireless channel $\overline{h}_{\alpha\beta}$ with $0.7$ radians standard deviation injected phase noise. (b) Reconstructed profile $F_{\alpha\beta}'(\phi,\theta)$ from reconstructed channel $h_{\alpha\beta}'$, with $0.5$ radians standard deviation phase noise for tolerance. Both tallest peaks align at $(\overline\phi = 45.6^{\circ},\overline\theta=90^{\circ})$. (c) and (d) are respective top views.
• Figure 2: Absolute AoA error from ground truth plotted against our AoA uncertainty metric. We see that nonzero AoA error grows as our AoA uncertainty metric grows, indicating that our metric successfully captures true error in measured AoA.
• Figure 3: The variance of the AoA error here as a function of AoA uncertainty is calculated empirically by finding the variance of the AoA error on the y-axis within a sliding window of $\Delta\kappa_{k,p} = 8.4\times 10^5$ along the x-axis. It is fit with a power curve of the form $y=ax^b$, with $r^2=0.8942$.
• Figure 4: PSNR improvement over epochs, with different setups.
• Figure 5: An example of the drone we reconstructed in the testbed. The left figure is the ground truth image, right figure is the re-rendered image from a trained model.