TACO: Temporal Consensus Optimization for Continual Neural Mapping

TL;DR

The paper addresses continual neural mapping in dynamic environments under strict memory and computation constraints by introducing TACO, a replay-free Temporal Consensus Optimization framework. TACO treats past model states as temporally weighted neighbors and enforces an importance-aware consensus to update the current map, enabling selective revision of outdated geometry while preserving stable regions. Built on Co-SLAM and ADMM-style optimization, it computes parameter importance from output sensitivity and uses this to regulate historical influence, achieving robust adaptation without data replay. Across static and dynamic simulations and real-world experiments, TACO consistently outperforms replay- and regularization-based baselines, delivering accurate, up-to-date maps with competitive memory usage and runtime, thus enabling scalable long-term neural mapping in dynamic robotic settings.

Abstract

Neural implicit mapping has emerged as a powerful paradigm for robotic navigation and scene understanding. However, real-world robotic deployment requires continual adaptation to changing environments under strict memory and computation constraints, which existing mapping systems fail to support. Most prior methods rely on replaying historical observations to preserve consistency and assume static scenes. As a result, they cannot adapt to continual learning in dynamic robotic settings. To address these challenges, we propose TACO (TemporAl Consensus Optimization), a replay-free framework for continual neural mapping. We reformulate mapping as a temporal consensus optimization problem, where we treat past model snapshots as temporal neighbors. Intuitively, our approach resembles a model consulting its own past knowledge. We update the current map by enforcing weighted consensus with historical representations. Our method allows reliable past geometry to constrain optimization while permitting unreliable or outdated regions to be revised in response to new observations. TACO achieves a balance between memory efficiency and adaptability without storing or replaying previous data. Through extensive simulated and real-world experiments, we show that TACO robustly adapts to scene changes, and consistently outperforms other continual learning baselines.

Figures (7)

• Figure 1: Comparison of continual neural mapping under environment changes. The yellow stool was moved from $A$ to $B$. Left: TACO (Temporal Consensus Optimization) revises outdated regions while preserving consistent geometry and yields an accurate and up-to-date map. Middle: Replay-based methods (e.g., Co-SLAM) preserve high-quality reconstructions overall, but their reliance on replayed past observations can lead to persistent "ghost" artifacts when scene geometry changes. Right: Regularization-based methods (e.g., MAS) over-constrain the optimization under conflicting evidence, which often results in distorted or fragmented reconstructions.
• Figure 2: Quantitative results on eight Replica scenes under static settings. We report reconstruction accuracy and completeness metrics, including Artifacts, Holes, Chamfer Distance, Completion Ratio, Precision@5cm, and F1@5cm. TACO achieves performance comparable to Co-SLAM with replay and matches or outperforms all continual learning baselines across all metrics.
• Figure 3: Qualitative reconstruction results on a representative Gibson scene.
• Figure 4: Additional hardware experiment with multiple sequential scene changes
• Figure 5: Quantitative Results on ScanNet