Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Image-Goal Navigation with Representative Latent World Model

Zhiwei Zhang, Hui Zhang, Kaihong Huang, Chenghao Shi, Huimin Lu

TL;DR

This work addresses image-goal navigation by learning a latent-world model that eschews pixel-level reconstruction in favor of high-level semantic representations. It introduces ReL-NWM, which encodes observations with DINOv3, fuses actions via FiLM, and uses history-aware spatiotemporal attention within a compact latent space to predict future states autoregressively. Training combines multi-step Cosine Similarity and MSE losses to enforce long-horizon consistency, while navigation is performed with Model Predictive Control using a cosine-based energy to a goal latent. Across simulated benchmarks and a real Unitree G1 robot with on-board planning, ReL-NWM achieves state-of-the-art trajectory prediction, faster planning, and robust real-world performance, demonstrating the practicality of latent-space planning for embodied navigation.

Abstract

World models enable robots to conduct counterfactual reasoning in physical environments by predicting future world states. While conventional approaches often prioritize pixel-level reconstruction of future scenes, such detailed rendering is computationally intensive and unnecessary for planning tasks like navigation. We therefore propose that prediction and planning can be efficiently performed directly within a latent space of high-level semantic representations. To realize this, we introduce the Representative Latent space Navigation World Model (ReL-NWM). Rather than relying on reconstructionoriented latent embeddings, our method leverages a pre-trained representation encoder, DINOv3, and incorporates specialized mechanisms to effectively integrate action signals and historical context within this representation space. By operating entirely in the latent domain, our model bypasses expensive explicit reconstruction and achieves highly efficient navigation planning. Experiments show state-of-the-art trajectory prediction and image-goal navigation performance on multiple benchmarks. Additionally, we demonstrate real-world applicability by deploying the system on a Unitree G1 humanoid robot, confirming its efficiency and robustness in practical navigation scenarios.

Efficient Image-Goal Navigation with Representative Latent World Model

TL;DR

This work addresses image-goal navigation by learning a latent-world model that eschews pixel-level reconstruction in favor of high-level semantic representations. It introduces ReL-NWM, which encodes observations with DINOv3, fuses actions via FiLM, and uses history-aware spatiotemporal attention within a compact latent space to predict future states autoregressively. Training combines multi-step Cosine Similarity and MSE losses to enforce long-horizon consistency, while navigation is performed with Model Predictive Control using a cosine-based energy to a goal latent. Across simulated benchmarks and a real Unitree G1 robot with on-board planning, ReL-NWM achieves state-of-the-art trajectory prediction, faster planning, and robust real-world performance, demonstrating the practicality of latent-space planning for embodied navigation.

Abstract

World models enable robots to conduct counterfactual reasoning in physical environments by predicting future world states. While conventional approaches often prioritize pixel-level reconstruction of future scenes, such detailed rendering is computationally intensive and unnecessary for planning tasks like navigation. We therefore propose that prediction and planning can be efficiently performed directly within a latent space of high-level semantic representations. To realize this, we introduce the Representative Latent space Navigation World Model (ReL-NWM). Rather than relying on reconstructionoriented latent embeddings, our method leverages a pre-trained representation encoder, DINOv3, and incorporates specialized mechanisms to effectively integrate action signals and historical context within this representation space. By operating entirely in the latent domain, our model bypasses expensive explicit reconstruction and achieves highly efficient navigation planning. Experiments show state-of-the-art trajectory prediction and image-goal navigation performance on multiple benchmarks. Additionally, we demonstrate real-world applicability by deploying the system on a Unitree G1 humanoid robot, confirming its efficiency and robustness in practical navigation scenarios.

Paper Structure

This paper contains 13 sections, 15 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Our Approach
  4. Representative Latent State Embedding
  5. Transition Model
  6. Training Policy
  7. Image-goal navigation with ReL-NWM
  8. Experiment Valuation
  9. Experiments on Datasets
  10. Ablation Study on Datasets
  11. Visual Navigation Performance
  12. Experiments on Real-World Humanoid Robot
  13. Conclusion

Figures (2)

  • Figure 1: Overview of the Representative Latent Navigation World Model (ReL-NWM). Our world model comprises two core modules: a World State Embedding that encodes images and robot actions into latent state representations, and a Transition Model that predicts future latent states based on the embedded world state. The model is trained autoregressively using a combination of MSE and Cosine Similarity losses.
  • Figure 2: Using the proposed ReL-NWM on a Unitree G1, the robot navigates from a start position to a user-specified goal Image. The figure compares the robot's onboard view (top row) with its external motion (bottom row). The successful arrival (Arrived Image) highlights the model's robustness to real-world visual noise and its efficiency on onboard hardware.