History-Conditioned Spatio-Temporal Visual Token Pruning for Efficient Vision-Language Navigation

TL;DR

This work proposes a training-free spatio-temporal vision token pruning framework tailored to VLA-based VLN, and successfully preserves superior navigation accuracy under extreme pruning scenarios, all while maintaining the highly competitive inference efficiency.

Abstract

Vision-Language Navigation (VLN) enables robots to follow natural-language instructions in visually grounded environments, serving as a key capability for embodied robotic systems. Recent Vision-Language-Action (VLA) models have demonstrated strong navigation performance, but their high computational cost introduces latency that limits real-time deployment. We propose a training-free spatio-temporal vision token pruning framework tailored to VLA-based VLN. We apply spatial token selection to the current view, alongside spatio-temporal compression for historical memories, enabling efficient long-horizon inference while reducing redundant computation. Leveraging attention-based token importance and query-guided spatio-temporal filtering, the proposed approach preserves navigation-relevant information without retraining or modifying pretrained models, allowing plug-and-play integration into existing VLA systems. Through experiments on standard VLN benchmarks, we confirm that our method significantly outperforms existing pruning strategies. It successfully preserves superior navigation accuracy under extreme pruning scenarios, all while maintaining the highly competitive inference efficiency. Real-world deployment on a Unitree Go2 quadruped robot further validates reliable and low-latency instruction-following navigation under practical robotic constraints. We hope this work helps bridge the gap between large-scale multimodal modeling and efficient, real-time embodied deployment in robotic navigation systems.

Figures (3)

• Figure 1: Performance stability and efficiency–performance dynamics of different vision token pruning methods. (a) SPL performance on the R2R benchmark across pruning ratios from 70% to 90%. While all methods experience performance degradation under more aggressive pruning, our method consistently maintains higher SPL than others (SparseVLM, DivPrune, and VisPruner), demonstrating superior preservation of critical visual information. (b) Performance–efficiency analysis at 90% pruning ratio. The horizontal axis shows inference throughput (FPS), and the vertical axis shows SPL performance. Bubble size represents the reduction in CUDA inference latency (ms) compared to the unpruned model, where larger bubbles indicate greater efficiency gains. Our method achieves both the highest throughput and strongest performance while maintaining competitive latency reduction, ultimately delivering a superior performance-efficiency profile.
• Figure 2: Overview of our proposed token pruning framework. Given a natural-language instruction and visual observations (history frames and the current frame), our framework proceeds in four stages. (A) Feature extraction and importance computation: All frames are encoded by a vision encoder, and the base importance $I_{\text{base}}$ of each patch token is computed from the attention weights of the global [CLS] token (normalized with Eq. \ref{['formula:I_base']}). (B) Token selection (current): We apply Adaptive Maximal Marginal Relevance (A-MMR) to select current-frame tokens by jointly considering base importance (i.e. semantics) and spatial diversity $(1-\textit{sim})$. The selected current tokens are used as queries $Q$. (C) Token selection (history): We again apply A-MMR to select history tokens based on importance and diversity, additionally guided by spatio-temporal relevance conditioned on the queries $Q$ from the current frame. (D) Action prediction: The selected tokens are fed into a projector and an LLM to predict the final navigation action sequence (e.g., $\uparrow \rightarrow \uparrow \uparrow$).
• Figure 3: Sample qualitative results demonstrating successful deployment of the StreamVLN model with and without token pruning on a Unitree Go2 quadruped across Outdoor, Workspace, and Lab environments. Note that the landmarks are marked in red, and the last frame of each sequence represents the VLN-commanded stop location.