GeNIE: A Generalizable Navigation System for In-the-Wild Environments

TL;DR

GeNIE tackles the challenge of generalizable outdoor navigation for embodied agents operating under diverse terrains, weather, and sensing configurations. The approach pairs a SAM-TP traversability predictor with a BEV-based path fusion planner that clusters and merges candidate paths to achieve stable, goal-aligned navigation, even under high latency. A 15,347-frame traversability dataset and a held-out benchmark across 18 countries support robust generalization, with ERC results showing 79% of the maximum score and zero human interventions. The work advances real-world navigation by delivering a scalable, open-resource framework including code, pretrained weights, and datasets that enable broader deployment and benchmarking.

Abstract

Reliable navigation in unstructured, real-world environments remains a significant challenge for embodied agents, especially when operating across diverse terrains, weather conditions, and sensor configurations. In this paper, we introduce GeNIE (Generalizable Navigation System for In-the-Wild Environments), a robust navigation framework designed for global deployment. GeNIE integrates a generalizable traversability prediction model built on SAM2 with a novel path fusion strategy that enhances planning stability in noisy and ambiguous settings. We deployed GeNIE in the Earth Rover Challenge (ERC) at ICRA 2025, where it was evaluated across six countries spanning three continents. GeNIE took first place and achieved 79% of the maximum possible score, outperforming the second-best team by 17%, and completed the entire competition without a single human intervention. These results set a new benchmark for robust, generalizable outdoor robot navigation. We will release the codebase, pretrained model weights, and newly curated datasets to support future research in real-world navigation.

Figures (6)

• Figure 1: Diverse input scenes (top), traversability predictions (middle), and final path planning results (bottom) across test environments unseen during training. GeNIE demonstrates strong generalization under challenging conditions, including heavy rain, high-contrast lighting, muddy lenses, and diverse terrain types.
• Figure 2: Examples of diverse terrains in ERC 2025, illustrating the requirement for robots to identify traversable areas in unstructured environments erc2025_video. Reproduced with permission from the ERC organizers.
• Figure 3: Overview of the GeNIE system. Given an RGB input image, the SAM-TP module predicts navigable regions in the image space. These predictions are projected into a bird's-eye view (BEV) cost map. Path fusion is then performed to identify coherent and safe traversable paths, followed by path selection based on alignment with the goal direction. Finally, the control module outputs linear and angular velocities to follow the selected path.
• Figure 4: Comparison of traversability predictions (red regions) from SAM, SAM2, VLM (gemini-2.5-flash), and our proposed approach. For SAM, a text prompt describing navigable regions is used directly. Since SAM2 does not support text prompts, we first query the VLM (gemini-2.5-flash) to identify relevant points in the image, which are then used as prompts for SAM2.
• Figure 5: Comparison of failure and success cases under different path planning strategies. The first and third rows show the robot’s current observation overlaid with the projected planned trajectory. The second and fourth rows display the BEV cost map with sampled paths; the selected path is shown in red, and the green dot at the edge of the map indicates the goal direction. Without path fusion, the planner tends to select paths with lower traversability when $\beta$ is small, or gets stuck in local minima when $\beta$ is large. In contrast, path fusion succeeds in both scenarios.