UrbanVLA: A Vision-Language-Action Model for Urban Micromobility

TL;DR

The paper addresses reliable, long-horizon navigation for urban micromobility using noisy route inputs from navigation apps. It proposes UrbanVLA, a route-conditioned Vision-Language-Action model that grounds roadbooks with visual observations to predict trajectory waypoints. A two-stage training pipeline—Supervised Fine-Tuning on simulated trajectories and web data, followed by Reinforcement Fine-Tuning with Implicit Q-Learning on sim-real data—enables robust, safe navigation. Experiments on MetaUrban show substantial improvements over baselines in SocialNav and PointNav, and real-world deployments confirm scalable, robust performance in dynamic urban environments.

Abstract

Urban micromobility applications, such as delivery robots, demand reliable navigation across large-scale urban environments while following long-horizon route instructions. This task is particularly challenging due to the dynamic and unstructured nature of real-world city areas, yet most existing navigation methods remain tailored to short-scale and controllable scenarios. Effective urban micromobility requires two complementary levels of navigation skills: low-level capabilities such as point-goal reaching and obstacle avoidance, and high-level capabilities, such as route-visual alignment. To this end, we propose UrbanVLA, a route-conditioned Vision-Language-Action (VLA) framework designed for scalable urban navigation. Our method explicitly aligns noisy route waypoints with visual observations during execution, and subsequently plans trajectories to drive the robot. To enable UrbanVLA to master both levels of navigation, we employ a two-stage training pipeline. The process begins with Supervised Fine-Tuning (SFT) using simulated environments and trajectories parsed from web videos. This is followed by Reinforcement Fine-Tuning (RFT) on a mixture of simulation and real-world data, which enhances the model's safety and adaptability in real-world settings. Experiments demonstrate that UrbanVLA surpasses strong baselines by more than 55% in the SocialNav task on MetaUrban. Furthermore, UrbanVLA achieves reliable real-world navigation, showcasing both scalability to large-scale urban environments and robustness against real-world uncertainties.

Figures (3)

• Figure 1: Overview of UrbanVLA. We collect diversified VideoQA data and urban micromobility demonstrations to train the model via a two-stage pipeline. In the SFT stage, UrbanVLA learns essential urban navigation capabilities such as goal-reaching, collision avoidance, and social compliance; in the RFT stage, we refine the model using a sim-real aggregated dataset with IQL, enhancing robustness in real-world scenarios.
• Figure 2: Real-world deployment of UrbanVLA. Our system consists of a quadruped robot equipped with GPS, Wi-Fi, a camera, and an onboard computing unit, along with a mobile-deployable console for real-time monitoring, sending navigation targets, visualizing maps and model predictions, and annotating teleoperation data used for reinforcement learning.
• Figure 3: Visualization of qualitative experiment results in real-world scenarios. We show four critical scenarios in real-world evaluation of UrbanVLA: overpass crossing, street turning, pedestrian crossing, and obstacle avoidance. UrbanVLA generates executive and reasonable trajectories shown by the blue trajectories plotted in the first person view (FPV).