\textsc{NaVIDA}: Vision-Language Navigation with Inverse Dynamics Augmentation

TL;DR

NaVIDA tackles VLN by explicitly modeling vision–action causality through inverse-dynamics augmentation. It combines Hierarchical Probabilistic Action Chunking (HPAC) with inverse-dynamics supervision (IDS) and an entropy-guided execution horizon to enable longer-horizon planning and reduce error accumulation, all within a lightweight 3B parameter framework. Empirical results on VLN-CE benchmarks show state-of-the-art or competitive performance with fewer parameters, and real-world robot experiments validate practical applicability. The approach offers a scalable, efficient paradigm for stabilizing language-grounded navigation in visually rich environments.

Abstract

Vision-and-Language Navigation (VLN) requires agents to interpret natural language instructions and act coherently in visually rich environments. However, most existing methods rely on reactive state-action mappings without explicitly modeling how actions causally transform subsequent visual observations. Lacking such vision-action causality, agents cannot anticipate the visual changes induced by its own actions, leading to unstable behaviors, weak generalization, and cumulative error along trajectory. To address these issues, we introduce \textsc{NaVIDA} (\textbf{Nav}igation with \textbf{I}nverse \textbf{D}ynamics \textbf{A}ugmentation), a unified VLN framework that couples policy learning with action-grounded visual dynamics and adaptive execution. \textsc{NaVIDA} augments training with chunk-based inverse-dynamics supervision to learn causal relationship between visual changes and corresponding actions. To structure this supervision and extend the effective planning range, \textsc{NaVIDA} employs hierarchical probabilistic action chunking (HPAC), which organizes trajectories into multi-step chunks and provides discriminative, longer-range visual-change cues. To further curb error accumulation and stabilize behavior at inference, an entropy-guided mechanism adaptively sets the execution horizon of action chunks. Extensive experiments show that \textsc{NaVIDA} achieves superior navigation performance compared to state-of-the-art methods with fewer parameters (3B vs. 8B). Real-world robot evaluations further validate the practical feasibility and effectiveness of our approach. Code and data will be available upon acceptance.

Figures (14)

• Figure 1: Comparision of three VLN paradigms. (a) Vanilla VLN relies on state-action mapping, lacking the causality between vision and actions; (b) Forward Dynamics Modeling (FDM) adds an additional image generation task which predicts future observations but is compute-heavy and distractible; (c) NaVIDA uses inverse dynamics supervision to learn the causality between vision and actions with the action predicition task only, making the learning process more efficient and stable.
• Figure 2: Architecture of NaVIDA. A multimodal language-model backbone outputs HPAC action blocks; training jointly optimizes navigation and IDM to strengthen the causal link between visual change and action, while inference uses entropy-guided execution horizon truncation to limit error accumulation.
• Figure 3: IDM data construction pipeline. From Habitat trajectories, HPAC groups atomic actions into variable-length blocks and induces waypoints; start–end frames yield (current, goal) views, and the intervening actions are ground-truth labels.
• Figure 4: Qualitative results of NaVIDA on VLN-CE R2R Val-Unseen split. Correct actions are marked with green arrows, while incorrect actions are marked with red arrows.
• Figure 5: Real world results of NaVIDA. The robot's direction of motion is indicated by arrows in the diagram.