Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VDRive: Leveraging Reinforced VLA and Diffusion Policy for End-to-end Autonomous Driving

Ziang Guo, Zufeng Zhang

TL;DR

VDRive tackles robustness in end-to-end autonomous driving by unifying reinforced Vision-Language-Action (VLA) representations with a diffusion-policy head. It employs CVQ-VAE to tokenize sensor observations and iteratively refines actions through offline RL and a critic, achieving contextual and geometric alignment between perception and control. Key contributions include reinforced modality alignment, a CVQ-VAE tokenization pipeline, and a dynamics-guided refinement module that yields hierarchical actions and refined trajectories. On nuScenes and Bench2Drive benchmarks, VDRive achieves state-of-the-art open-loop and closed-loop performance, e.g., $L_2$ error $=0.29$ m and collision rate $=18\%$ on nuScenes, and Driving Score $=66.25$, Success Rate $=50.51\%$ on Bench2Drive, demonstrating strong planning accuracy and safety in dynamic driving scenarios.

Abstract

In autonomous driving, dynamic environment and corner cases pose significant challenges to the robustness of ego vehicle's state understanding and decision making. We introduce VDRive, a novel pipeline for end-to-end autonomous driving that explicitly models state-action mapping to address these challenges, enabling interpretable and robust decision making. By leveraging the advancement of the state understanding of the Vision Language Action Model (VLA) with generative diffusion policy-based action head, our VDRive guides the driving contextually and geometrically. Contextually, VLA predicts future observations through token generation pre-training, where the observations are represented as discrete codes by a Conditional Vector Quantized Variational Autoencoder (CVQ-VAE). Geometrically, we perform reinforcement learning fine-tuning of the VLA to predict future trajectories and actions based on current driving conditions. VLA supplies the current state tokens and predicted state tokens for the action policy head to generate hierarchical actions and trajectories. During policy training, a learned critic evaluates the actions generated by the policy and provides gradient-based feedback, forming an actor-critic framework that enables a reinforcement-based policy learning pipeline. Experiments show that our VDRive achieves state-of-the-art performance in the Bench2Drive closed-loop benchmark and nuScenes open-loop planning.

VDRive: Leveraging Reinforced VLA and Diffusion Policy for End-to-end Autonomous Driving

TL;DR

VDRive tackles robustness in end-to-end autonomous driving by unifying reinforced Vision-Language-Action (VLA) representations with a diffusion-policy head. It employs CVQ-VAE to tokenize sensor observations and iteratively refines actions through offline RL and a critic, achieving contextual and geometric alignment between perception and control. Key contributions include reinforced modality alignment, a CVQ-VAE tokenization pipeline, and a dynamics-guided refinement module that yields hierarchical actions and refined trajectories. On nuScenes and Bench2Drive benchmarks, VDRive achieves state-of-the-art open-loop and closed-loop performance, e.g., error m and collision rate on nuScenes, and Driving Score , Success Rate on Bench2Drive, demonstrating strong planning accuracy and safety in dynamic driving scenarios.

Abstract

In autonomous driving, dynamic environment and corner cases pose significant challenges to the robustness of ego vehicle's state understanding and decision making. We introduce VDRive, a novel pipeline for end-to-end autonomous driving that explicitly models state-action mapping to address these challenges, enabling interpretable and robust decision making. By leveraging the advancement of the state understanding of the Vision Language Action Model (VLA) with generative diffusion policy-based action head, our VDRive guides the driving contextually and geometrically. Contextually, VLA predicts future observations through token generation pre-training, where the observations are represented as discrete codes by a Conditional Vector Quantized Variational Autoencoder (CVQ-VAE). Geometrically, we perform reinforcement learning fine-tuning of the VLA to predict future trajectories and actions based on current driving conditions. VLA supplies the current state tokens and predicted state tokens for the action policy head to generate hierarchical actions and trajectories. During policy training, a learned critic evaluates the actions generated by the policy and provides gradient-based feedback, forming an actor-critic framework that enables a reinforcement-based policy learning pipeline. Experiments show that our VDRive achieves state-of-the-art performance in the Bench2Drive closed-loop benchmark and nuScenes open-loop planning.
Paper Structure (17 sections, 15 equations, 4 figures, 4 tables)

This paper contains 17 sections, 15 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. VLA
  4. Diffusion Policy
  5. Dataset Construction
  6. Pre-training and RL Fine-tuning of VLA
  7. Diffusion Policy Head Training
  8. Proposed Methodology
  9. Preliminary
  10. Reinforced Modality Alignment
  11. Dynamics-guided Refinement
  12. Experiments
  13. Open-loop Experiments
  14. Closed-loop Experiments
  15. Qualitative Results
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Comparison of VLA-based autonomous driving.
  • Figure 2: Example of our reinforcement learning fine-tuning dataset.
  • Figure 3: Proposed VDRive framework. A reinforced framework empowering VLA’s prediction of segmented drivable future and policy heads' generation of hierarchical actions and trajectories. With reinforcement learning and discrete representation of sensor data, VDRive is trained to align context and geometry tokens in a low dimensional space.
  • Figure 4: Visualization of the refined trajectory predictions across simulated, real-world's dark, and rainy scenes.