Path Planning using Instruction-Guided Probabilistic Roadmaps

TL;DR

IG-PRM addresses instruction-driven path planning for mobile robots by converting natural language instructions into embedding vectors that condition a learned cost map derived from occupancy maps. A neural cost predictor combines instruction embeddings with occupancy data to produce an instruction-guided cost map, which guides PRM sampling and edge costs; a shortest-path search then yields the instruction-aligned path. The method is trained with supervised data from ground-truth cost maps and evaluated on synthetic and real-world 2D indoor environments, showing improved path optimality and instruction adherence compared to standard PRM. The approach preserves modularity by acting as a drop-in enhancement for global planners and scales to multi-query settings, with potential extensions to 3D and integration with other planners.

Abstract

This work presents a novel data-driven path planning algorithm named Instruction-Guided Probabilistic Roadmap (IG-PRM). Despite the recent development and widespread use of mobile robot navigation, the safe and effective travels of mobile robots still require significant engineering effort to take into account the constraints of robots and their tasks. With IG-PRM, we aim to address this problem by allowing robot operators to specify such constraints through natural language instructions, such as ``aim for wider paths'' or ``mind small gaps''. The key idea is to convert such instructions into embedding vectors using large-language models (LLMs) and use the vectors as a condition to predict instruction-guided cost maps from occupancy maps. By constructing a roadmap based on the predicted costs, we can find instruction-guided paths via the standard shortest path search. Experimental results demonstrate the effectiveness of our approach on both synthetic and real-world indoor navigation environments.

Figures (7)

• Figure 1: Instruction-Guided Path Planning. Given natural language instructions such as "choose wider passage," our planner finds a collision-free path from the start to the goal while taking into account the instructions.
• Figure 2: Schematic Diagram of IG-PRM. (1) Convert natural language instructions into embeddings. (2) Combine occupancy maps with embeddings to predict instruction-guided cost maps, where the cost prediction network is trained in a supervised fashion. (3) Use the predicted cost maps for producing instruction-guided paths.
• Figure 3: Qualitative Results on Synthetic Instances. Roadmap and solution paths are visualized with gray and black lines, respectively. Obstacle regions are colored in blue. The start and goal points are marked with green and red circles, respectively.
• Figure 4: Qualitative Results on IN2D Instances. Roadmap and solution paths are visualized with gray and black lines, respectively. Obstacle regions are colored in blue. The start and goal points are marked with green and red circles, respectively.
• Figure 5: Quantitative Results on Synthetic Instances. We evaluated success rates weighted by path lengths (SPL; the higher, the better) as well as dynamic-time-warping distances (DTW; the lower, the better) against ground-truth paths for each method while varying the number of nodes.