VLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness

TL;DR

This work proposes VLN-Cache, a visual-dynamic-aware and semantic-dynamic-aware caching framework that introduces view-aligned remapping to recover geometric correspondences and a task-relevance saliency filter to veto reuse at semantic transitions.

Abstract

Vision-and-Language Navigation (VLN) increasingly relies on large vision-language models, but their inference cost conflicts with real-time deployment. Token caching is a promising training-free strategy that avoids redundant computation by reusing stable visual tokens across frames. However, existing methods assume a static camera and fixed semantic focus, assumptions that VLN fundamentally violates. We identify two failure modes: (1) visual dynamics, where viewpoint shift displaces token positions across frames, causing position-wise matching to pair misaligned content; (2) semantic dynamics, where token relevance shifts across task stages as navigation progresses, making cached states stale. We propose VLN-Cache, a visual-dynamic-aware and semantic-dynamic-aware caching framework that introduces view-aligned remapping to recover geometric correspondences and a task-relevance saliency filter to veto reuse at semantic transitions. A layer-adaptive entropy policy further balances the per-layer reuse budget. Experiments on the R2R-CE simulation benchmark show up to 1.52x speedup while maintaining competitive navigation success rates.

Figures (9)

• Figure 1: Overview of the Proposed VLN-Cache Framework.
• Figure 2: Visual and semantic dynamics along a VLN trajectory.
• Figure 3: Position-wise token mismatch under viewpoint shift.
• Figure 4: Overview of the VLN-Cache framework.
• Figure 5: System implementation pipeline of VLN-Cache.