CausalNav: A Long-term Embodied Navigation System for Autonomous Mobile Robots in Dynamic Outdoor Scenarios

TL;DR

This work tackles open-world, language-guided navigation for autonomous robots in dynamic outdoor environments. It introduces CausalNav, a hierarchical Embodied Graph framework built from LLMs and offline maps, enabling retrieval-augmented reasoning and semantic planning for long-horizon tasks. Key contributions include a multi-level Embodied Graph that fuses coarse map data with fine-grained objects, a spatio-temporal corridor for dynamic object filtering, and language-driven planning with open-vocabulary queries, all updated online. Experiments in both simulation and real-campus settings demonstrate robustness, efficiency, and superior long-range navigation performance, including edge-enabled operation without relying on cloud APIs.

Abstract

Autonomous language-guided navigation in large-scale outdoor environments remains a key challenge in mobile robotics, due to difficulties in semantic reasoning, dynamic conditions, and long-term stability. We propose CausalNav, the first scene graph-based semantic navigation framework tailored for dynamic outdoor environments. We construct a multi-level semantic scene graph using LLMs, referred to as the Embodied Graph, that hierarchically integrates coarse-grained map data with fine-grained object entities. The constructed graph serves as a retrievable knowledge base for Retrieval-Augmented Generation (RAG), enabling semantic navigation and long-range planning under open-vocabulary queries. By fusing real-time perception with offline map data, the Embodied Graph supports robust navigation across varying spatial granularities in dynamic outdoor environments. Dynamic objects are explicitly handled in both the scene graph construction and hierarchical planning modules. The Embodied Graph is continuously updated within a temporal window to reflect environmental changes and support real-time semantic navigation. Extensive experiments in both simulation and real-world settings demonstrate superior robustness and efficiency.

Figures (7)

• Figure 1: The overall workflow of CausalNav. CausalNav introduces a novel embodied navigation framework that integrates open-vocabulary semantic reasoning, dynamic environment adaptation, and Embodied Graph-based planning. By dynamically updating the navigation graph with both historical and real-time data, the system enables robust, long-horizon, and language-directed navigation in complex outdoor environments.
• Figure 2: The CausalNav framework comprises three sequential modules: (1) Open-vocabulary Object Tracking and Ego-motion Estimation(in Sec. \ref{['Open-vocabulary Object Tracking and Ego-motion Estimation']}): Integrates RGB, LiDAR, and IMU inputs for open-vocabulary object detection and tracking, along with ego-motion estimation via 2D–3D spatial-temporal alignment. (2) Dynamic Object Filtering and Embodied Graph Construction(in Sec. \ref{['retrival']}): Filters transient dynamic objects through asynchronous multi-object tracking and constructs a temporally stable embodied scene graph. (3) Embodied Graph Updating and Human Language Navigation(in Sec. \ref{['Embodied Graph Updating and Language-guided Planning']}): Builds and updates a semantic graph using LLMs to interpret language commands and perform hierarchical planning, with real-time dynamic object removal for robust navigation.
• Figure 3: Illustration of three observed trajectory points and their corresponding 3D bounding boxes within the spatial-temporal corridor. The point cloud depicts the same vehicle captured at different timestamps.
• Figure 4: The simulation environment and the constructed Embodied Graph. The environment includes coarse-grained objects (e.g., buildings) and fine-grained ones (e.g., fire hydrants, mailboxes). The Embodied Graph fuses both levels and updates dynamically as the agent moves.
• Figure 5: The robot used in real-world navigation experiments.