RAMEN: Real-time Asynchronous Multi-agent Neural Implicit Mapping

TL;DR

The paper tackles real-time multi-agent neural implicit mapping under irregular communications by introducing RAMEN, which combines an uncertainty-weighted decentralized C-ADMM with a frequency-based epistemic uncertainty measure. By per-parameter weighting of consensus terms, RAMEN biases information fusion toward more reliable updates, achieving robust, asynchronous collaboration across agents. The approach is validated through extensive simulations on Replica and ScanNet datasets and real hardware experiments with Turtlebots, showing improved completion, reduced artifacts, and resilience to low communication rates compared to state-of-the-art baselines. This work enables practical, scalable multi-robot neural mapping in real-world networks and demonstrates the first real-world deployment of asynchronous multi-agent neural implicit mapping.

Abstract

Multi-agent neural implicit mapping allows robots to collaboratively capture and reconstruct complex environments with high fidelity. However, existing approaches often rely on synchronous communication, which is impractical in real-world scenarios with limited bandwidth and potential communication interruptions. This paper introduces RAMEN: Real-time Asynchronous Multi-agEnt Neural implicit mapping, a novel approach designed to address this challenge. RAMEN employs an uncertainty-weighted multi-agent consensus optimization algorithm that accounts for communication disruptions. When communication is lost between a pair of agents, each agent retains only an outdated copy of its neighbor's map, with the uncertainty of this copy increasing over time since the last communication. Using gradient update information, we quantify the uncertainty associated with each parameter of the neural network map. Neural network maps from different agents are brought to consensus on the basis of their levels of uncertainty, with consensus biased towards network parameters with lower uncertainty. To achieve this, we derive a weighted variant of the decentralized consensus alternating direction method of multipliers (C-ADMM) algorithm, facilitating robust collaboration among agents with varying communication and update frequencies. Through extensive evaluations on real-world datasets and robot hardware experiments, we demonstrate RAMEN's superior mapping performance under challenging communication conditions.

Figures (8)

• Figure 1: In a challenging real-world experiment with limited communication (agents can only exchange information every 30 seconds), our method RAMEN enables each turtlebot to successfully map the full scene while only physically visiting half of the scene (explored areas and trajectories are colored accordingly). Our method achieves accuracy comparable to the ground truth while the baseline method (DiNNO) fails to converge.
• Figure 2: This visualization shows the uncertainty associated with the parameters in the coarsest feature grid $V_\alpha^{l=1}$ of the neural implicit map. We represent this uncertainty by overlaying color-coded vertices on the reconstructed 3D mesh. The robot's trajectory is shown in red. The robot's trajectory and field-of-view are limited to the left half of the room. As expected, high uncertainty (blue vertices) corresponds to the right half of the map not explored by the robot where the reconstructed geometry is inaccurate.
• Figure 3: Comparison of scene reconstructions from DiNNO and RAMEN (ours) on ScanNet "scene0000" with only 50% communication success rate. RAMEN produces more accurate and detailed geometry in the highlighted region.
• Figure 4: Reconstruction quality of DiNNO and RAMEN (ours) on ScanNet "scene0169" under varying communication success rates (three-agent trajectories shown in different colors). Even at a 20% success rate, RAMEN preserves scene details, while DiNNO exhibits incomplete or blurry reconstructions.
• Figure 5: Evaluation metric scores for DiNNO and RAMEN (ours) on ScanNet "scene0169" under varying communication success rates. While both methods exhibit worse scores with lower communication rate, RAMEN demonstrates greater robustness, maintaining higher accuracy and lower variance than DiNNO, especially in lower communication rates.