Semantic Zone-Based Map Management for Stable AI-Integrated Mobile Robots

Abstract

Recent advances in large AI models (VLMs and LLMs) and joint use of the 3D dense maps, enable mobile robots to provide more powerful and interactive services grounded in rich spatial context. However, deploying both heavy AI models and dense maps on edge robots is challenging under strict memory budgets. When the memory budget is exceeded, required keyframes may not be loaded in time, which can degrade the stability of position estimation and interfering model performance. We proposes a semantic zone-based map management approach to stabilize dense-map utilization under memory constraints. We associate keyframes with semantic indoor regions (e.g., rooms and corridors) and keyframe management at the semantic zone level prioritizes spatially relevant map content while respecting memory constraints. This reduces keyframe loading and unloading frequency and memory usage. We evaluate the proposed approach in large-scale simulated indoor environments and on an NVIDIA Jetson Orin Nano under concurrent SLAM-VLM execution. With Qwen3.5:0.8b, the proposed method improves throughput by 3.3 tokens/s and reduces latency by 21.7% relative to a geometric map-management strategy. Furthermore, while the geometric strategy suffers from out-of-memory failures and stalled execution under memory pressure, the proposed method eliminates both issues, preserving localization stability and enabling robust VLM operation. These results demonstrate that the proposed approach enables efficient dense map utilization for memory constrained, AI-integrated mobile robots. Code is available at: https://github.com/huichangs/rtabmap/tree/segment

Figures (7)

• Figure 3: Overview of the Semantic Zone Based Map Management
• Figure 4: Overview of semantic zone-based keyframe management. The Semantic Zone Manager uses the estimated pose to determine the current zone and to form an active keyframe list. The Memory Manager enforces MemoryThr by maintaining an active zone list and an unload zone list, ensuring strict compliance with the memory limit before loading new keyframes from the map database.
• Figure 5: Memory usage comparison between (a) Basic + VLM and (b) Semantic + VLM.
• Figure 6: Trajectory comparison among GT, Basic only, Basic+VLM, and Semantic+VLM. The blue circle indicates the final accepted loop-closure position in the Basic+VLM run. After this point, no additional loop closures are accepted until the end of the run.
• Figure 7: Distributions of VLM token generation throughput and end-to-end latency (Semantic + gemma3:4b).