VL-LN Bench: Towards Long-horizon Goal-oriented Navigation with Active Dialogs

TL;DR

The paper tackles the challenge of long-horizon goal-oriented navigation with vague instructions by introducing Interactive Instance Object Navigation (IION) and the VL-LN benchmark. It presents an automatic data-generation pipeline and an oracle-based evaluation framework enabling online dialog-enabled navigation; experiments show that proactive querying improves both IION and ION performance, achieving state-of-the-art results on the benchmark. Key findings indicate that image–attribute grounding, exploration efficiency, and query strategy remain bottlenecks, with humans outperforming agents in dialog efficiency. The VL-LN benchmark provides a scalable resource for training and evaluating dialog-enabled embodied navigation in realistic house-scale environments.

Abstract

In most existing embodied navigation tasks, instructions are well-defined and unambiguous, such as instruction following and object searching. Under this idealized setting, agents are required solely to produce effective navigation outputs conditioned on vision and language inputs. However, real-world navigation instructions are often vague and ambiguous, requiring the agent to resolve uncertainty and infer user intent through active dialog. To address this gap, we propose Interactive Instance Object Navigation (IION), a task that requires agents not only to generate navigation actions but also to produce language outputs via active dialog, thereby aligning more closely with practical settings. IION extends Instance Object Navigation (ION) by allowing agents to freely consult an oracle in natural language while navigating. Building on this task, we present the Vision Language-Language Navigation (VL-LN) benchmark, which provides a large-scale, automatically generated dataset and a comprehensive evaluation protocol for training and assessing dialog-enabled navigation models. VL-LN comprises over 41k long-horizon dialog-augmented trajectories for training and an automatic evaluation protocol with an oracle capable of responding to agent queries. Using this benchmark, we train a navigation model equipped with dialog capabilities and show that it achieves significant improvements over the baselines. Extensive experiments and analyses further demonstrate the effectiveness and reliability of VL-LN for advancing research on dialog-enabled embodied navigation. Code and dataset: https://0309hws.github.io/VL-LN.github.io/

Figures (4)

• Figure 1: A case for the IION task. The oracle (top left) first gives a simple goal-oriented navigation instruction ("Search for the chair."). The agent has to locate a specific instance of the given category (chair). The agent can ask three types of questions—attribute, route, and disambiguation—to progressively resolve ambiguity and locate the target (instance). The full description in the bottom right is the instruction given to the agent in the ION task, which can locate the specific chair in this environment.
• Figure 2: Automatic pipeline for collecting dialog-augmented trajectories. We first aggregate room-level instance attributes into unified house-level annotations. We then pair each target instance with a start point to generate episodes. Finally, we collect dialog-augmented trajectories using a frontier-based exploration (FBE) agent that, with 90% probability, selects the frontier nearest to the previously chosen frontier and, with 10%, selects the frontier closest to the target (the “best frontier”). The attribute question is asked at the beginning of the trajectory, and the attribute is randomly chosen from one of the given attributes shown in the figure. The route question is asked when the best frontier is chosen. And the disambiguation question is proposed when an instance with the same category as the target is detected, the criterion of "detected" is that the GT semantic appears in the center of the image, and the instance is within 3 meters of the agent. The number following "#" indicates the corresponding number of cases. The ellipses indicate the potential inclusion of additional disambiguation, route, or attribute questions.
• Figure 3: Trajectory statistics of the VL–LN dataset. (a–b) Frequency histograms of per-episode path steps, and dialog turns; (c) frequency histogram of per-turn dialog length (tokens). Black lines denote smoothed density fits. (d) Nested donut of dialog data. Outer ring: target-instance category proportions; inner disk: question-type proportions (Attribute, Disambiguation, and Route question).
• Figure 4: Failure cases. Green curves denote the geodesic shortest paths; blue curves are the navigator’s exploration trajectories; red shaded regions indicate the success zone around the target. (a) Referential ambiguity: within the same view, the navigator and the Oracle refer to different instances, causing the navigator to stop at a wrong instance. (b) Partial observability: the navigator only observes a single candidate in the room and stops without disambiguating. (c) Exploration failure: despite continued interaction, the human navigator never finds the target.