Active Test-time Vision-Language Navigation

TL;DR

The paper addresses vision-language navigation under distribution shifts at test time by introducing ATENA, a framework that integrates active human feedback with self-supervised signals to calibrate uncertainty during online navigation. ATENA combines Mixture Entropy Optimization, which conditions entropy on episodic outcomes using a mixture of the policy and a pseudo-expert distribution, with Self-Active Learning, guided by an uncertainty-based query strategy and a self-prediction head. The approach updates the policy with a joint objective that balances feedback-driven and self-supervised signals, and selectively queries feedback to maximize information while minimizing labeling cost. Empirical results on REVERIE, R2R, and R2R-CE show that ATENA outperforms strong baselines and existing TTA methods, improving robustness to test-time distribution shifts and uncertainty calibration. Overall, the work provides a scalable framework for interactive and autonomous test-time adaptation in VLN and related embodied AI tasks.

Abstract

Vision-Language Navigation (VLN) policies trained on offline datasets often exhibit degraded task performance when deployed in unfamiliar navigation environments at test time, where agents are typically evaluated without access to external interaction or feedback. Entropy minimization has emerged as a practical solution for reducing prediction uncertainty at test time; however, it can suffer from accumulated errors, as agents may become overconfident in incorrect actions without sufficient contextual grounding. To tackle these challenges, we introduce ATENA (Active TEst-time Navigation Agent), a test-time active learning framework that enables a practical human-robot interaction via episodic feedback on uncertain navigation outcomes. In particular, ATENA learns to increase certainty in successful episodes and decrease it in failed ones, improving uncertainty calibration. Here, we propose mixture entropy optimization, where entropy is obtained from a combination of the action and pseudo-expert distributions-a hypothetical action distribution assuming the agent's selected action to be optimal-controlling both prediction confidence and action preference. In addition, we propose a self-active learning strategy that enables an agent to evaluate its navigation outcomes based on confident predictions. As a result, the agent stays actively engaged throughout all iterations, leading to well-grounded and adaptive decision-making. Extensive evaluations on challenging VLN benchmarks-REVERIE, R2R, and R2R-CE-demonstrate that ATENA successfully overcomes distributional shifts at test time, outperforming the compared baseline methods across various settings.

Figures (7)

• Figure 1: Overview of the ATENA adaptation framework. At each navigation step, the agent stores state and entropy information in its memory. Once the episode ends, the stored entropy is used to determine the feedback source: human oracle for uncertain episodes, and self oracle for certain episodes. Self oracle utilizes a self-prediction head, trained during online test-time, enabling the agent to autonomously predict navigation success or failure by itself.
• Figure 2: llustration of Mixture Entropy Optimization (MEO). (a) The Mixture Action Distribution is constructed by combining the action distribution (yellow) with a pseudo-expert distribution (red). (b) Mixture entropy is minimized for successful episodes to encourage the correct actions, and maximized for failures to penalize incorrect ones.
• Figure 3: Effect of the combination weight $\lambda$. Performance comparison with different combination weight $\lambda$ in Mixture Entropy Optimization. $\lambda=0$ corresponds to vanilla entropy without distribution mix. The results are averaged across three experiments with different seeds.
• Figure 4: Different Episode Sampling Strategies. Our uncertainty-based sampling outperforms baselines and remains competitive against full-feedback settings.
• Figure 5: In Step 3, Trial 1 selected an incorrect action with 94.5% confidence. After ATENA's adaptation, Trial 2 correctly choose the appropriate action with 99.9% confidence. The intended target—the faucet in the bathroom with a dark green hand towel—was missed in Trial 1 (final distance: 10.56), but successfully located in Trial 2.