Schrödinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation

TL;DR

This work tackles zero-shot object navigation under severe occlusions by introducing Schrödinger’s Navigator, which imagines multiple plausible futures along three candidate trajectories using a trajectory-conditioned 3D world model. Imagined futures are metrically aligned, semantically annotated, and fused into a future-aware value map that, together with a multi-sourced navigation map, guides a planner toward safer, less uncertain routes and better target tracking. An LLM-based goal grounding combined with 3D imagination yields a compact, actionable affordance map without task-specific retraining, improving performance in occlusion-heavy real-world scenarios and showing strong DTG and SR in simulation. Real-world experiments on a Go2 robot demonstrate robust improvements over strong baselines, especially in dynamic and obstacle-rich settings, while Habitat simulations corroborate the approach’s effectiveness in diverse indoor environments. Overall, explicit trajectory-conditioned 3D imagination enables robust, uncertainty-aware zero-shot navigation with practical implications for service robotics and embodied AI in real-world, cluttered spaces.

Abstract

Zero-shot object navigation (ZSON) requires a robot to locate a target object in a previously unseen environment without relying on pre-built maps or task-specific training. However, existing ZSON methods often struggle in realistic and cluttered environments, particularly when the scene contains heavy occlusions, unknown risks, or dynamically moving target objects. To address these challenges, we propose \textbf{Schrödinger's Navigator}, a navigation framework inspired by Schrödinger's thought experiment on uncertainty. The framework treats unobserved space as a set of plausible future worlds and reasons over them before acting. Conditioned on egocentric visual inputs and three candidate trajectories, a trajectory-conditioned 3D world model imagines future observations along each path. This enables the agent to see beyond occlusions and anticipate risks in unseen regions without requiring extra detours or dense global mapping. The imagined 3D observations are fused into the navigation map and used to update a value map. These updates guide the policy toward trajectories that avoid occlusions, reduce exposure to uncertain space, and better track moving targets. Experiments on a Go2 quadruped robot across three challenging scenarios, including severe static occlusions, unknown risks, and dynamically moving targets, show that Schrödinger's Navigator consistently outperforms strong ZSON baselines in self-localization, object localization, and overall Success Rate in occlusion-heavy environments. These results demonstrate the effectiveness of trajectory-conditioned 3D imagination in enabling robust zero-shot object navigation.

Figures (9)

• Figure 1: Real-world zero-shot object navigation often fails when the target object (e.g., a cat) is hidden behind occlusions and surrounded by unknown or potentially hazardous space. Conventional navigation systems typically perceive only the immediate occluder and are unable to infer what exists beyond it. Our Schrödinger’s Navigator addresses this challenge by modeling the unobserved regions as multiple plausible futures. It explicitly samples several trajectories around the occluding structure and uses a trajectory-conditioned 3DGS imagination model to predict the expected observations along each path. This allows the robot to anticipate the post-occlusion scene and select safer, less-occluded routes that increase the likelihood of locating the target.
• Figure 2: Overview of our Navigator pipeline. Left: The system receives a goal instruction, RGB-D observations, and the robot pose as input. Bottom center: A trajectory sampler deterministically selects three candidate trajectories and conditions a 3D world model. The model predicts future 3DGS observations along these trajectories—left bypass, right bypass, and over-the-top—to infer occluded and unobserved regions. Top right: The predicted cues are fused with current observations to construct and update multi-sourced value maps and enable future-aware reasoning. This process produces a final affordance map used for intermediate waypoint selection. Bottom right: The execution unit follows the selected waypoint and generates control commands to navigate the robot continuously toward the goal.
• Figure 3: Sampling the camera trajectory around the obstacle to maximize field of view coverage.
• Figure 4: Overview of trajectory-conditioned 3D world model. Given the current RGB frame and the initial robot pose, a trajectory sampler produces a discrete set of three candidate camera trajectories (left, right, up). These trajectories are then used to condition a 3D world model that predicts a 3DGS scene for each candidate. From the predicted scenes, we render short RGB videos and their corresponding depth maps. The rendered depths are then aligned with the current depth observation to estimate a global scale factor $s$, which is subsequently used to consistently scale and align the predicted 3DGS scenes. The aligned scenes are finally transformed into the world coordinate frame and fused into a single merged 3DGS scene that is geometrically and visually consistent with the robot’s current observation.
• Figure 5: System setup for experiment.