NAMO-LLM: Efficient Navigation Among Movable Obstacles with Large Language Model Guidance

TL;DR

NAMO-LLM tackles the challenging NAMO problem by integrating a non-uniform, LLM-guided sampling strategy into a sampling-based planner, enabling efficient planning in environments with many movable obstacles. The approach preserves probabilistic completeness and demonstrates substantial runtime improvements and higher-quality plans compared to baselines and pure LLM planners. Key contributions include the LLM-guided sampling framework, a prompting scheme to elicit obstacle relocation recommendations, and rigorous probabilistic completeness proofs. Practical validation shows strong performance in simulation and a real-world TurtleBot3 hardware demonstration, suggesting significant impact for robot manipulation in cluttered, dynamic environments.

Abstract

Several planners have been proposed to compute robot paths that reach desired goal regions while avoiding obstacles. However, these methods fail when all pathways to the goal are blocked. In such cases, the robot must reason about how to reconfigure the environment to access task-relevant regions - a problem known as Navigation Among Movable Objects (NAMO). While various solutions to this problem have been developed, they often struggle to scale to highly cluttered environments. To address this, we propose NAMO-LLM, a sampling-based planner that searches over robot and obstacle configurations to compute feasible plans specifying which obstacles to move, where, and in what order. Its key novelty is a non-uniform sampling strategy guided by Large Language Models (LLMs) biasing the tree construction toward directions more likely to yield a solution. We show that NAMO-LLM is probabilistically complete and demonstrate through experiments that it efficiently scales to cluttered environments, outperforming related works in both runtime and plan quality.

Figures (6)

• Figure 1: A robot must reach a destination behind the building block. Since all direct paths are blocked, it relocates obstacles to access the goal region.
• Figure 2: Tree structure built by the proposed planner. Each node (blue) represents a state ${\mathbf q}(t)$, and each edge (green arrow) represents a manipulation action. For instance, ${\mathbf q}_{\text{new}}$ is reached from ${\mathbf q}_{\text{rand}}$ via action $a({\mathbf c}_5, {\mathbf c}_5')$ moving obstacle $o_5$. If moving $o_5$ is recommended by the LLM (rather than random sampling), the score $v({\mathbf q}_{\text{new}})$ is set to $1$; otherwise, it remains $0$. The label next to each node indicates whether it was generated by moving an LLM-suggested obstacle (L) or a randomly selected obstacle (R) (see Section \ref{['M2']}). The number next to the label is the score $u({\mathbf q})$ yielding a set ${\mathcal{V}}_v$ that collects the top two gray-shaded nodes (see Section \ref{['M2']}). The two snapshots illustrate the environment associated with ${\mathbf q}_{\text{rand}}$ and ${\mathbf q}_{\text{new}}$.
• Figure 3: Illustration of the components ${\mathcal{N}}(t)$ (left) and ${\mathcal{N}}^5(t)$ (right). Uniform vs non-uniform sampling for obstacle selection: In the left figure, only obstacles 1, 3, 4, and 5 are manipulable. Under uniform sampling, each obstacle $o_i$ will be selected with probability $p_i$ defined as $p_1=p_3=p_4=p_5=0.25$ and $p_2=0$. Under non-uniform sampling, if the LLM-recommended obstacle $o_{\text{LLM}}$ is $o_3$ and $p_{\text{obs}}=0.8$, then the corresponding probabilities become $p_3=0.85$, $p_1=p_4=p_5=0.05$, and $p_2=0$.
• Figure 4: Input prompt for GPT-4o and $K=1$ in Case Study V (Sec. \ref{['sec:sims']}).
• Figure 5: Initial configurations of the obstacles (red polygons) and robot for Case Studies 1-4. The green path connects the initial robot state to the first obstacle to be repositioned for clearing a path to the goal area ${\mathcal{P}}_g$.

Theorems & Definitions (1)

• Theorem 3.1