VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memory

TL;DR

VLingNav tackles the problem of brittle generalization and poor long-horizon planning in embodied navigation by integrating adaptive reasoning (AdaCoT) and persistent cross-modal memory (VLingMem) within a Vision-Language-Action framework. The approach is supported by Nav-AdaCoT-2.9M, a large, reasoning-annotated dataset, and an online expert-guided RL post-training regime that blends exploration with expert oversight. Empirical results show state-of-the-art performance across ObjectNav, EVT, and ImageNav benchmarks, with successful zero-shot transfer to real-world robots. The work demonstrates that recursive reasoning and durable linguistic memory can markedly improve robustness, efficiency, and cross-domain generalization in embodied agents, advancing the practicality of cognitive VLA systems for real-world navigation tasks.

Abstract

VLA models have shown promising potential in embodied navigation by unifying perception and planning while inheriting the strong generalization abilities of large VLMs. However, most existing VLA models rely on reactive mappings directly from observations to actions, lacking the explicit reasoning capabilities and persistent memory required for complex, long-horizon navigation tasks. To address these challenges, we propose VLingNav, a VLA model for embodied navigation grounded in linguistic-driven cognition. First, inspired by the dual-process theory of human cognition, we introduce an adaptive chain-of-thought mechanism, which dynamically triggers explicit reasoning only when necessary, enabling the agent to fluidly switch between fast, intuitive execution and slow, deliberate planning. Second, to handle long-horizon spatial dependencies, we develop a visual-assisted linguistic memory module that constructs a persistent, cross-modal semantic memory, enabling the agent to recall past observations to prevent repetitive exploration and infer movement trends for dynamic environments. For the training recipe, we construct Nav-AdaCoT-2.9M, the largest embodied navigation dataset with reasoning annotations to date, enriched with adaptive CoT annotations that induce a reasoning paradigm capable of adjusting both when to think and what to think about. Moreover, we incorporate an online expert-guided reinforcement learning stage, enabling the model to surpass pure imitation learning and to acquire more robust, self-explored navigation behaviors. Extensive experiments demonstrate that VLingNav achieves state-of-the-art performance across a wide range of embodied navigation benchmarks. Notably, VLingNav transfers to real-world robotic platforms in a zero-shot manner, executing various navigation tasks and demonstrating strong cross-domain and cross-task generalization.

Figures (12)

• Figure 1: Overview of VLingNav. VLingNav is a VLA model enhanced with adaptive CoT reasoning and visual-assisted linguistic memory. This architecture allows the model to leverage historical visual and linguistic memory, achieving SOTA results on several embodied navigation benchmarks. Furthermore, VLingNav can be deployed zero-shot on real-world robots to perform diverse and complex navigation tasks.
• Figure 2: The overall framework of VLingNav. The framework takes video streams and multimodal instruction as input to produce robot action for navigation with tailored linguistic designs. AdaCoT can adaptively generate linguistic thinking according to its observation, while VLingMem summarizes CoT cues with key visual features for globally informed decision-making.
• Figure 3: Data distribution and instruction word cloud for the VLingNav training dataset.
• Figure 4: The autonomous adaptive CoT labeling pipeline of VLingNav.
• Figure 5: Online post-training with a hybrid rollout procedure.