Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IROS: A Dual-Process Architecture for Real-Time VLM-Based Indoor Navigation

Joonhee Lee, Hyunseung Shin, Jeonggil Ko

TL;DR

This work tackles the challenge of real-time indoor navigation with semantic understanding by integrating Vision-Language Models into a dual-process framework. IROS decouples fast reflexive perception (System One) from slow deliberative reasoning (System Two), using conditional inference, patch-based scene changes, and spatial-textual augmentation to keep latency low on-device. The approach yields substantial gains: up to 66% reduction in travel time and improved decision fidelity when compared with VLM-only baselines, alongside robust on-device operation on a Jetson platform. The findings demonstrate that compact VLMs, when aided by structured spatial and textual cues and selective VLM invocation, can achieve human-like navigation with real-time responsiveness in diverse indoor environments. The work suggests future extensions with reinforcement learning-based fine control and broader applicability to manipulation tasks.

Abstract

Indoor mobile robot navigation requires fast responsiveness and robust semantic understanding, yet existing methods struggle to provide both. Classical geometric approaches such as SLAM offer reliable localization but depend on detailed maps and cannot interpret human-targeted cues (e.g., signs, room numbers) essential for indoor reasoning. Vision-Language-Action (VLA) models introduce semantic grounding but remain strictly reactive, basing decisions only on visible frames and failing to anticipate unseen intersections or reason about distant textual cues. Vision-Language Models (VLMs) provide richer contextual inference but suffer from high computational latency, making them unsuitable for real-time operation on embedded platforms. In this work, we present IROS, a real-time navigation framework that combines VLM-level contextual reasoning with the efficiency of lightweight perceptual modules on low-cost, on-device hardware. Inspired by Dual Process Theory, IROS separates fast reflexive decisions (System One) from slow deliberative reasoning (System Two), invoking the VLM only when necessary. Furthermore, by augmenting compact VLMs with spatial and textual cues, IROS delivers robust, human-like navigation with minimal latency. Across five real-world buildings, IROS improves decision accuracy and reduces latency by 66% compared to continuous VLM-based navigation.

IROS: A Dual-Process Architecture for Real-Time VLM-Based Indoor Navigation

TL;DR

This work tackles the challenge of real-time indoor navigation with semantic understanding by integrating Vision-Language Models into a dual-process framework. IROS decouples fast reflexive perception (System One) from slow deliberative reasoning (System Two), using conditional inference, patch-based scene changes, and spatial-textual augmentation to keep latency low on-device. The approach yields substantial gains: up to 66% reduction in travel time and improved decision fidelity when compared with VLM-only baselines, alongside robust on-device operation on a Jetson platform. The findings demonstrate that compact VLMs, when aided by structured spatial and textual cues and selective VLM invocation, can achieve human-like navigation with real-time responsiveness in diverse indoor environments. The work suggests future extensions with reinforcement learning-based fine control and broader applicability to manipulation tasks.

Abstract

Indoor mobile robot navigation requires fast responsiveness and robust semantic understanding, yet existing methods struggle to provide both. Classical geometric approaches such as SLAM offer reliable localization but depend on detailed maps and cannot interpret human-targeted cues (e.g., signs, room numbers) essential for indoor reasoning. Vision-Language-Action (VLA) models introduce semantic grounding but remain strictly reactive, basing decisions only on visible frames and failing to anticipate unseen intersections or reason about distant textual cues. Vision-Language Models (VLMs) provide richer contextual inference but suffer from high computational latency, making them unsuitable for real-time operation on embedded platforms. In this work, we present IROS, a real-time navigation framework that combines VLM-level contextual reasoning with the efficiency of lightweight perceptual modules on low-cost, on-device hardware. Inspired by Dual Process Theory, IROS separates fast reflexive decisions (System One) from slow deliberative reasoning (System Two), invoking the VLM only when necessary. Furthermore, by augmenting compact VLMs with spatial and textual cues, IROS delivers robust, human-like navigation with minimal latency. Across five real-world buildings, IROS improves decision accuracy and reduces latency by 66% compared to continuous VLM-based navigation.
Paper Structure (37 sections, 22 figures, 1 table)

This paper contains 37 sections, 22 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. RL and VLMs as Decision Makers
  4. Other Vision Models as Decision Makers
  5. On-Device Deployment Challenges
  6. Theoretical Foundation: Dual Process Theory
  7. Empirical Motivation: Addressing VLM Reasoning Redundancy
  8. IROS Design Overview
  9. Detailed Design of IROS
  10. System One: Fast Perception/Matching
  11. Key Frame Compare
  12. Spatial and Textual Understanding
  13. Condition Matching
  14. System Two: Deliberative Reasoning and Augmentation
  15. Offline Condition-to-Action Generation
  16. ...and 22 more sections

Figures (22)

  • Figure 1: Conceptual example of IROS with semantic-level navigational input, "go to room 304."
  • Figure 2: Example scenario of making unnecessary repetitive VLM calls on hallways.
  • Figure 3: Overview of IROS workflow.
  • Figure 4: Detailed overview of System One
  • Figure 5: Spatial & Textual information of the Scene
  • ...and 17 more figures