TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environments

TL;DR

TIC-VLA tackles the fundamental problem of mismatch between slow semantic reasoning and fast real-time control in language-guided robot navigation. It introduces a latency-aware framework that decouples thinking and acting via a delayed semantic-control interface, and trains policies to robustly interpret delayed semantic signals through a three-stage latency-consistent pipeline. The approach is validated in a physics-based, dynamic-people simulation suite (DynaNav) and on real robots, showing consistent gains over prior VLA methods and robustness to multi-second reasoning latency. This work enables more reliable edge deployment of language-conditioned navigation in human-centric environments and lays groundwork for extending latency-aware reasoning to broader robotics tasks.

Abstract

Robots in dynamic, human-centric environments must follow language instructions while maintaining real-time reactive control. Vision-language-action (VLA) models offer a promising framework, but they assume temporally aligned reasoning and control, despite semantic inference being inherently delayed relative to real-time action. We introduce Think-in-Control (TIC)-VLA, a latency-aware framework that explicitly models delayed semantic reasoning during action generation. TIC-VLA defines a delayed semantic-control interface that conditions action generation on delayed vision-language semantic states and explicit latency metadata, in addition to current observations, enabling policies to compensate for asynchronous reasoning. We further propose a latency-consistent training pipeline that injects reasoning inference delays during imitation learning and online reinforcement learning, aligning training with asynchronous deployment. To support realistic evaluation, we present DynaNav, a physics-accurate, photo-realistic simulation suite for language-guided navigation in dynamic environments. Extensive experiments in simulation and on a real robot show that TIC-VLA consistently outperforms prior VLA models while maintaining robust real-time control under multi-second reasoning latency. Project website: https://ucla-mobility.github.io/TIC-VLA/

Figures (12)

• Figure 1: TIC-VLA enables real-time, language-conditioned navigation by decoupling slow vision-language reasoning from fast reactive control via a delayed semantic-control interface. A latency-consistent training strategy improves robustness under variable reasoning delays. Performance is demonstrated in the DynaNav simulation and real-world indoor and outdoor navigation tasks.
• Figure 2: Overview of TIC-VLA. The architecture adopts a decoupled dual-system design with a fast action expert and a slow reasoning VLM. A shared vision encoder provides real-time observations to the policy and time-lagged observations to the VLM, where the delay arises naturally from slow inference. The delayed semantic-control interface (including delayed VLM KV cache features and latency metadata) is explicitly recorded. The Transformer-based action expert takes as input the current observation, robot state, and delayed semantic-control interface data to generate actions from learnable action queries via cross-attention. Multi-stage training combines imitation learning with delayed inference and reinforcement learning to ensure robustness to real-world, time-sensitive conditions.
• Figure 3: Details of TIC-VLA action policy structure, training, and asynchronous execution. (a) Latency-aware action policy that predicts action chunks from multimodal inputs. (b) Value network used during online reinforcement learning. (c) Three-stage latency-consistent training pipeline combining VLM supervision, imitation learning, and reinforcement learning. (d) Asynchronous inference and control with explicit latency modeling.
• Figure 4: Qualitative results of TIC-VLA closed-loop performance in DynaNav hospital (top) and office (bottom) environments.
• Figure 5: The effect of VLM asynchronous reasoning inference latency in TIC-VLA on task performance.