Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?

Siqi Wang, Chao Liang, Yunfan Gao, Erxin Yu, Sen Li, Yushi Li, Jing Li, Haofen Wang

TL;DR

CitySeeker introduces the first large-scale benchmark for embodied urban navigation driven by implicit human needs, using 6,440 trajectories across 8 cities and 7 need categories to probe long-horizon reasoning and groundable visual search. The paper reveals significant gaps in current VLMs' ability to translate abstract intents into concrete, multi-step plans, and proposes a triad of human-inspired strategies—Backtracking, Cognitive-map Enrichment, and Memory-Based Retrieval (BCR)—to boost spatial intelligence. Through comprehensive evaluations of 27 models and extensive analyses of error modes and city biases, the work highlights both the limits of current systems and the potential of memory and structured spatial cues to improve last-mile navigation. The findings offer a concrete roadmap for building spatially aware, memory-informed VLMs capable of robustly addressing implicit needs in dynamic urban environments.

Abstract

Vision-Language Models (VLMs) have made significant progress in explicit instruction-based navigation; however, their ability to interpret implicit human needs (e.g., "I am thirsty") in dynamic urban environments remains underexplored. This paper introduces CitySeeker, a novel benchmark designed to assess VLMs' spatial reasoning and decision-making capabilities for exploring embodied urban navigation to address implicit needs. CitySeeker includes 6,440 trajectories across 8 cities, capturing diverse visual characteristics and implicit needs in 7 goal-driven scenarios. Extensive experiments reveal that even top-performing models (e.g., Qwen2.5-VL-32B-Instruct) achieve only 21.1% task completion. We find key bottlenecks in error accumulation in long-horizon reasoning, inadequate spatial cognition, and deficient experiential recall. To further analyze them, we investigate a series of exploratory strategies-Backtracking Mechanisms, Enriching Spatial Cognition, and Memory-Based Retrieval (BCR), inspired by human cognitive mapping's emphasis on iterative observation-reasoning cycles and adaptive path optimization. Our analysis provides actionable insights for developing VLMs with robust spatial intelligence required for tackling "last-mile" navigation challenges.

CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?

TL;DR

CitySeeker introduces the first large-scale benchmark for embodied urban navigation driven by implicit human needs, using 6,440 trajectories across 8 cities and 7 need categories to probe long-horizon reasoning and groundable visual search. The paper reveals significant gaps in current VLMs' ability to translate abstract intents into concrete, multi-step plans, and proposes a triad of human-inspired strategies—Backtracking, Cognitive-map Enrichment, and Memory-Based Retrieval (BCR)—to boost spatial intelligence. Through comprehensive evaluations of 27 models and extensive analyses of error modes and city biases, the work highlights both the limits of current systems and the potential of memory and structured spatial cues to improve last-mile navigation. The findings offer a concrete roadmap for building spatially aware, memory-informed VLMs capable of robustly addressing implicit needs in dynamic urban environments.

Abstract

Vision-Language Models (VLMs) have made significant progress in explicit instruction-based navigation; however, their ability to interpret implicit human needs (e.g., "I am thirsty") in dynamic urban environments remains underexplored. This paper introduces CitySeeker, a novel benchmark designed to assess VLMs' spatial reasoning and decision-making capabilities for exploring embodied urban navigation to address implicit needs. CitySeeker includes 6,440 trajectories across 8 cities, capturing diverse visual characteristics and implicit needs in 7 goal-driven scenarios. Extensive experiments reveal that even top-performing models (e.g., Qwen2.5-VL-32B-Instruct) achieve only 21.1% task completion. We find key bottlenecks in error accumulation in long-horizon reasoning, inadequate spatial cognition, and deficient experiential recall. To further analyze them, we investigate a series of exploratory strategies-Backtracking Mechanisms, Enriching Spatial Cognition, and Memory-Based Retrieval (BCR), inspired by human cognitive mapping's emphasis on iterative observation-reasoning cycles and adaptive path optimization. Our analysis provides actionable insights for developing VLMs with robust spatial intelligence required for tackling "last-mile" navigation challenges.

Paper Structure

This paper contains 55 sections, 4 equations, 19 figures, 15 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. CitySeeker Benchmark
  4. Benchmark Construction
  5. Evaluation on CitySeeker
  6. Overview
  7. Evaluation Setup
  8. Main Results and Analysis
  9. Overall Performance
  10. Task Category Analysis.
  11. Variation Across Cities.
  12. In-Depth Analysis
  13. Trajectory Patterns.
  14. Exploratory Approaches to Embodied Urban Navigation
  15. Backtracking Mechanisms
  16. ...and 40 more sections

Figures (19)

  • Figure 1: Navigation Instructions indicating explicit needs (top) and implicit needs (bottom).
  • Figure 2: The statistics of CitySeeker with 6,440 trajectories in diverse scenarios.
  • Figure 3: Construction Process of the CitySeeker Dataset.
  • Figure 4: The CitySeeker Implicit-Need-Driven Embodied Urban Navigation Framework.
  • Figure 5: Radar Chart: TCP performance of different models across various subcategories. Bar Chart: TCP performance of models in different cities (left: New York, right: Beijing).
  • ...and 14 more figures