NoRD: A Data-Efficient Vision-Language-Action Model that Drives without Reasoning

TL;DR

This work identifies that standard Group Relative Policy Optimization (GRPO) fails to yield significant improvements when applied to policies trained on such small, reasoning-free datasets, and overcomes this by incorporating Dr.~GRPO, a recent algorithm designed to mitigate difficulty bias in LLMs.

Abstract

Vision-Language-Action (VLA) models are advancing autonomous driving by replacing modular pipelines with unified end-to-end architectures. However, current VLAs face two expensive requirements: (1) massive dataset collection, and (2) dense reasoning annotations. In this work, we address both challenges with NORD (No Reasoning for Driving). Compared to existing VLAs, NORD achieves competitive performance while being fine-tuned on <60% of the data and no reasoning annotations, resulting in 3x fewer tokens. We identify that standard Group Relative Policy Optimization (GRPO) fails to yield significant improvements when applied to policies trained on such small, reasoning-free datasets. We show that this limitation stems from difficulty bias, which disproportionately penalizes reward signals from scenarios that produce high-variance rollouts within GRPO. NORD overcomes this by incorporating Dr. GRPO, a recent algorithm designed to mitigate difficulty bias in LLMs. As a result, NORD achieves competitive performance on Waymo and NAVSIM with a fraction of the training data and no reasoning overhead, enabling more efficient autonomous systems. Website: https://nord-vla-ai.github.io/

Figures (13)

• Figure 1: Comparison of VLA training pipelines. (a) Existing approaches depend on large-scale reasoning data generation, followed by extensive SFT and RL fine-tuning. (b) In contrast, NoRD directly utilizes a small-scale driving dataset for SFT, and performs RL fine-tuning tailored for weak SFT policy, enabling data-efficient learning without reasoning supervision.
• Figure 2: Reward distribution in the weak SFT model. The group-mean PDM score is shown with band representing the mean of the corresponding group standard deviation for NoRD-base. GRPO struggles to optimize high-variance regions (the majority) and is effective only in low-variance regions (the trajectories in green and red are for ground truth and NoRD-base prediction).
• Figure 3: Evolution of group-mean PDM score during RL fine-tuning. (a) GRPO struggles to optimize samples with high group variance during training, particularly in the range $[0.2$–$0.65]$. (b) Dr. GRPO effectively optimizes high-variance samples during training, resulting in significant overall performance gains.
• Figure 4: Qualitative comparison of RL fine-tuning (RLFT) on the weak SFT model using GRPO and Dr. GRPO. With Dr. GRPO, NoRD successfully learns complex maneuvers such as sharp turns and lane changes without collisions, whereas GRPO fails to optimize the weak SFT model (NoRD-base) and collides (in red).
• Figure 5: Model architecture of NoRD.NoRD directly predicts action tokens without requiring reasoning traces, enabling a significantly more efficient training and inference pipeline.