Follow the Signs: Using Textual Cues and LLMs to Guide Efficient Robot Navigation

TL;DR

The paper tackles efficient semantic navigation in unfamiliar indoor environments by using textual cues and room-numbering patterns to bias exploration. It introduces a hybrid framework that fuses local perception with LLM-based semantic inferences into a confidence grid, guiding planning via frontier exploration and A*. Key contributions include an LLM-driven goal-extraction module, a belief-like confidence grid with exponential decay that updates from semantic predictions, and a demonstration across seven environments showing near-optimal paths. Quantitatively, the method achieves an SPL of $0.745$, beating Frontier Exploration ($0.596$), NavGPT ($0.236$), and LLM-Only ($0.160$), with paths more than 40% shorter, and a real-world Spot deployment demonstrates practical viability. This work demonstrates how grounding LLMs in structured spatial cues and integrating them with classical planning can enable efficient semantic navigation in large, partially observable indoor spaces.

Abstract

Autonomous navigation in unfamiliar environments often relies on geometric mapping and planning strategies that overlook rich semantic cues such as signs, room numbers, and textual labels. We propose a novel semantic navigation framework that leverages large language models (LLMs) to infer patterns from partial observations and predict regions where the goal is most likely located. Our method combines local perceptual inputs with frontier-based exploration and periodic LLM queries, which extract symbolic patterns (e.g., room numbering schemes and building layout structures) and update a confidence grid used to guide exploration. This enables robots to move efficiently toward goal locations labeled with textual identifiers (e.g., "room 8") even before direct observation. We demonstrate that this approach enables more efficient navigation in sparse, partially observable grid environments by exploiting symbolic patterns. Experiments across environments modeled after real floor plans show that our approach consistently achieves near-optimal paths and outperforms baselines by over 25% in Success weighted by Path Length.

Figures (4)

• Figure 1: Overview of navigation pipeline. At each timestep, the agent updates its seen grids from local observations. If the goal is directly visible, it plans a shortest path to it using A*. Otherwise, the LLM predicts a likely goal region, the confidence grid is updated, and the agent plans a path either toward the highest-confidence region or, if the confidence grid is uniform, toward the nearest unexplored frontier. This loop repeats until the goal is found or the environment is fully explored.
• Figure 2: Examples of grid environments. Black and white cells denote occupancy grid values, with black indicating obstacles and white indicating free space. Blue cells represent the locations of signs. Left: Small H-Shape environment; Middle: Large Offices environment; Right: Noisy Polycam environment.
• Figure 3: Evolution of the agent’s grids over time for the goal of "find room 641L". The top row shows a fusion of the seen semantic and seen occupancy grids with blue dots representing signs and blue text representing room numbers. The bottom row shows the confidence grid with higher confidence indicating likely target locations. The grid is initialized with all zeros. As the LLM predicts likely goal directions, confidence values increase in corresponding regions, while explored cells inconsistent with the goal are reset to zero. Over time, repeated predictions with exponential decay lead to smooth gradients of confidence that guide exploration toward the most likely goal region (Step 27).
• Figure 4: Real-world deployment of our method on Spot. The robot is tasked with reaching room 621. Upon encountering an initial sign, Spot infers that room numbers likely increase from left to right and chooses to explore left. It then detects another directional sign confirming the goal’s location and successfully navigates to the target room. Note: the target room was labeled as 621 instead of 45-621 in the ground truth grid world.