AstraNav-World: World Model for Foresight Control and Consistency

TL;DR

<3-5 sentence high-level summary> AstraNav-World tackles the problem of catastrophic error accumulation in long-horizon embodied navigation by unifying foresight vision and control within a single probabilistic world model. It uses a Vision-Language Model planner to condition both a diffusion-based video generator and an action policy head, enabling synchronized rollouts and bidirectional constraints that enforce physical consistency. The approach includes a 3D-RoPE rearrangement for multi-view inputs and Sparse Foresight Scheduling to balance accuracy and speed, with two policy options (Action Former and Diffusion Policy) trained end-to-end. On R2R-CE, RxR-CE, and HM3D-OVON, AstraNav-World achieves strong gains and demonstrates notable zero-shot transfer to real-world robots without additional fine-tuning.

Abstract

Embodied navigation in open, dynamic environments demands accurate foresight of how the world will evolve and how actions will unfold over time. We propose AstraNav-World, an end-to-end world model that jointly reasons about future visual states and action sequences within a unified probabilistic framework. Our framework integrates a diffusion-based video generator with a vision-language policy, enabling synchronized rollouts where predicted scenes and planned actions are updated simultaneously. Training optimizes two complementary objectives: generating action-conditioned multi-step visual predictions and deriving trajectories conditioned on those predicted visuals. This bidirectional constraint makes visual predictions executable and keeps decisions grounded in physically consistent, task-relevant futures, mitigating cumulative errors common in decoupled "envision-then-plan" pipelines. Experiments across diverse embodied navigation benchmarks show improved trajectory accuracy and higher success rates. Ablations confirm the necessity of tight vision-action coupling and unified training, with either branch removal degrading both prediction quality and policy reliability. In real-world testing, AstraNav-World demonstrated exceptional zero-shot capabilities, adapting to previously unseen scenarios without any real-world fine-tuning. These results suggest that AstraNav-World captures transferable spatial understanding and planning-relevant navigation dynamics, rather than merely overfitting to simulation-specific data distribution. Overall, by unifying foresight vision and control within a single generative model, we move closer to reliable, interpretable, and general-purpose embodied agents that operate robustly in open-ended real-world settings.

Figures (3)

• Figure 1: Overview of AstraNav World architecture. Our model adapts to two different policy heads within a unified framework: (a) a traditional policy head with an Action Former for direct action prediction, and (b) a diffusion policy head for probabilistic action generation. These two policy streams share a common VLM planner ($\tau_{\theta}$), which processes instructions and visual history to generate high-level conditional tokens. The architecture also includes a VLM conditional video generator ($\upsilon_{\theta}$) for predicting and planning consistent future visual scenes. When using diffusion strategy flow, multimodal fusion cross attention (MMFCA) can be selectively used in overlapping DiTBlocks to interconnect, achieving bidirectional information flow between action and visual prediction.
• Figure 2: Qualitative results. Our model predict the next five visual frames while simultaneously outputting the corresponding 5-step waypoint sequence. The generated frames are in strong agreement with the scenes rendered from the predicted waypoints and also the predicted trajectory marked with red arrows, showing strong consistency.
• Figure 3: Ablation study results. (a) Impact of the video generator (VG): Comparing the Success Rate (SR) with and without VG across R2R, RxR, and OVON datasets shows that predicting future observations consistently improves navigation performance. (b) Efficiency of SFS: Evaluation of the proposed SFS strategy on R2R with different skipping intervals $k$. The results demonstrate that increasing $k$ significantly reduces inference time (up to 6.7$\times$ speedup) while maintaining robust performance.