Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LGR: LLM-Guided Ranking of Frontiers for Object Goal Navigation

Mitsuaki Uno, Kanji Tanaka, Daiki Iwata, Yudai Noda, Shoya Miyazaki, Kouki Terashima

TL;DR

This work tackles mapless Object Goal Navigation by reframing frontier visit ordering as an LLM-driven ranking problem. The LGR framework uses a view-to-prompt strategy and Reciprocal Rank Features to fuse multi-view observations, enabling zero-shot reasoning about frontier priority within a single view and across updates. A modular system combines Mask R-CNN-based perception, occupancy-grid planning, and A* navigation, guided by LLM-derived subgoals and frontier rankings. Evaluations in Habitat-Sim with HM3D demonstrate that LGR consistently improves exploration efficiency (SPL) over random frontier baselines, highlighting the practical impact of integrating commonsense reasoning into robot exploration. Overall, LGR offers a flexible, generalizable approach to frontier selection in unknown environments, reducing unnecessary travel while maintaining robust object-goal search performance.

Abstract

Object Goal Navigation (OGN) is a fundamental task for robots and AI, with key applications such as mobile robot image databases (MRID). In particular, mapless OGN is essential in scenarios involving unknown or dynamic environments. This study aims to enhance recent modular mapless OGN systems by leveraging the commonsense reasoning capabilities of large language models (LLMs). Specifically, we address the challenge of determining the visiting order in frontier-based exploration by framing it as a frontier ranking problem. Our approach is grounded in recent findings that, while LLMs cannot determine the absolute value of a frontier, they excel at evaluating the relative value between multiple frontiers viewed within a single image using the view image as context. We dynamically manage the frontier list by adding and removing elements, using an LLM as a ranking model. The ranking results are represented as reciprocal rank vectors, which are ideal for multi-view, multi-query information fusion. We validate the effectiveness of our method through evaluations in Habitat-Sim.

LGR: LLM-Guided Ranking of Frontiers for Object Goal Navigation

TL;DR

This work tackles mapless Object Goal Navigation by reframing frontier visit ordering as an LLM-driven ranking problem. The LGR framework uses a view-to-prompt strategy and Reciprocal Rank Features to fuse multi-view observations, enabling zero-shot reasoning about frontier priority within a single view and across updates. A modular system combines Mask R-CNN-based perception, occupancy-grid planning, and A* navigation, guided by LLM-derived subgoals and frontier rankings. Evaluations in Habitat-Sim with HM3D demonstrate that LGR consistently improves exploration efficiency (SPL) over random frontier baselines, highlighting the practical impact of integrating commonsense reasoning into robot exploration. Overall, LGR offers a flexible, generalizable approach to frontier selection in unknown environments, reducing unnecessary travel while maintaining robust object-goal search performance.

Abstract

Object Goal Navigation (OGN) is a fundamental task for robots and AI, with key applications such as mobile robot image databases (MRID). In particular, mapless OGN is essential in scenarios involving unknown or dynamic environments. This study aims to enhance recent modular mapless OGN systems by leveraging the commonsense reasoning capabilities of large language models (LLMs). Specifically, we address the challenge of determining the visiting order in frontier-based exploration by framing it as a frontier ranking problem. Our approach is grounded in recent findings that, while LLMs cannot determine the absolute value of a frontier, they excel at evaluating the relative value between multiple frontiers viewed within a single image using the view image as context. We dynamically manage the frontier list by adding and removing elements, using an LLM as a ranking model. The ranking results are represented as reciprocal rank vectors, which are ideal for multi-view, multi-query information fusion. We validate the effectiveness of our method through evaluations in Habitat-Sim.

Paper Structure

This paper contains 18 sections, 5 equations, 9 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Configuration
  4. Mobile Robot Image Database (MRID)
  5. LGR Approach
  6. Reciprocal Rank Feature (RRF)
  7. Image Recognition Module
  8. Motion Planning Module
  9. Prompt Design
  10. Determining Orientation
  11. Determining Subgoals
  12. Selecting Movement Direction
  13. Moving Toward the Subgoal
  14. Experiment
  15. Dataset (Habitat Workspace)
  16. ...and 3 more sections

Figures (9)

  • Figure 1: The relationship between systems and modules. To avoid unnecessary complexity, common modules such as SLAM are omitted in this figure.
  • Figure 2: Examples of mask R-CNN outputs.
  • Figure 3: Prompt for room category classification
  • Figure 4: List of room categories
  • Figure 5: Prompt for ranking locations
  • ...and 4 more figures