T2Nav Algebraic Topology Aware Temporal Graph Memory and Loop Detection for ZeroShot Visual Navigation

TL;DR

T2Nav is introduced, a zeroshot navigation system that integrates heterogeneous data and employs graph-based reasoning and demonstrates flexibility by handling goals specified using reference images of target object instances, making it particularly suitable for scenarios in which agents must navigate to visually similar yet spatially distinct instances.

Abstract

Deploying autonomous agents in real world environments is challenging, particularly for navigation, where systems must adapt to situations they have not encountered before. Traditional learning approaches require substantial amounts of data, constant tuning, and, sometimes, starting over for each new task. That makes them hard to scale and not very flexible. Recent breakthroughs in foundation models, such as large language models and vision language models, enable systems to attempt new navigation tasks without requiring additional training. However, many of these methods only work with specific input types, employ relatively basic reasoning, and fail to fully exploit the details they observe or the structure of the spaces. Here, we introduce T2Nav, a zeroshot navigation system that integrates heterogeneous data and employs graph-based reasoning. By directly incorporating visual information into the graph and matching it to the environment, our approach enables the system to strike a good balance between exploration and goal attainment. This strategy allows robust obstacle avoidance, reliable loop closure detection, and efficient path planning while eliminating redundant exploration patterns. The system demonstrates flexibility by handling goals specified using reference images of target object instances, making it particularly suitable for scenarios in which agents must navigate to visually similar yet spatially distinct instances. Experiments demonstrate that our approach is efficient and adapts well to unknown environments, moving toward practical zero-shot instance-image navigation capabilities.

Figures (6)

• Figure 1: Conceptual overview of T$^2$-Nav. The framework addresses loop-closure problems in which the agent initially becomes trapped in repetitive exploration patterns. To address this issue, we propose two novel modules: (1) TeRM maintains cross-temporal object relationships, and (2) scene dynamics are explored through graph-based reasoning for TSLC to employ persistent homology, enabling the detection and avoidance of navigation loops via topological invariants of agent trajectories, thereby facilitating a robust zero-shot visual navigation system.
• Figure 2: Overview of $\text{T}^2$-Nav system. (a) Multi-modal inputs with GPS, pose, and goal image specifying the target instance, and RGBD input to construct a dynamic scene graph. (b) Graph Processing with RGBD images performs scene-goal matching to identify potential target instances, with ✓/$\times$ indicating success/failure. (c) Loop closure detection maintains a blacklist of locations. (d) TeRM provides temporal consistency for robust instance tracking. (d) The TeRM module maintains temporal consistency across consecutive scene graphs for robust instance tracking across varying viewpoints and environmental conditions. (e) The navigation pipeline generates occupancy maps, applies a deterministic local policy for obstacle avoidance, and outputs action decisions.
• Figure 3: TeRM Framework Overview. (a) Scene graphs track object persistence and new entities across timesteps. (b) Temporal edge construction for dependencies between nodes using exponentially decayed edge weights across $t$. (c) Physics-inspired positions prediction refines object trajectories based on motion dynamics.
• Figure 4: TSLC illustration: Agent trajectories are processed through Vietoris-Rips filtration to derive persistent homological features, which are subsequently mapped to a persistence diagram in a birth-death coordinate system. Loop closure is identified by assessing whether the $W_2$ distance between the persistence diagram of the current trajectory segment and those of historical segments falls below a predefined threshold.
• Figure 5: Qualitative comparison of frontier selection strategies between UniGoal and the proposed method. Compared with the baseline UniGoal, $\text{T}^2$-Nav selects frontiers more strategically, for more efficient exploration and reducing redundant path traversals.