DriveLaW:Unifying Planning and Video Generation in a Latent Driving World

TL;DR

DriveLaW addresses the long-standing gap between high-fidelity driving video generation and trajectory planning by unifying them in a latent world model. It chains a spatiotemporal video generator (DriveLaW-Video) with a diffusion-based planner (DriveLaW-Act) in a shared latent space and trains them via a three-stage curriculum to balance visual fidelity and planning reliability. The approach achieves state-of-the-art video generation metrics on nuScenes (FID $=4.6$, FVD $=81.3$) and sets a new record on NAVSIM planning (PDMS $=89.1$), demonstrating strong closed-loop performance without post-training or external scorers. By leveraging latent priors distilled from large-scale driving video, DriveLaW bridges perception and control, enabling more stable and semantically coherent autonomous driving in diverse scenarios.

Abstract

World models have become crucial for autonomous driving, as they learn how scenarios evolve over time to address the long-tail challenges of the real world. However, current approaches relegate world models to limited roles: they operate within ostensibly unified architectures that still keep world prediction and motion planning as decoupled processes. To bridge this gap, we propose DriveLaW, a novel paradigm that unifies video generation and motion planning. By directly injecting the latent representation from its video generator into the planner, DriveLaW ensures inherent consistency between high-fidelity future generation and reliable trajectory planning. Specifically, DriveLaW consists of two core components: DriveLaW-Video, our powerful world model that generates high-fidelity forecasting with expressive latent representations, and DriveLaW-Act, a diffusion planner that generates consistent and reliable trajectories from the latent of DriveLaW-Video, with both components optimized by a three-stage progressive training strategy. The power of our unified paradigm is demonstrated by new state-of-the-art results across both tasks. DriveLaW not only advances video prediction significantly, surpassing best-performing work by 33.3% in FID and 1.8% in FVD, but also achieves a new record on the NAVSIM planning benchmark.

Figures (7)

• Figure 1: Comparison of different world-model paradigms. Our DriveLaW serially connects the Video Model and Action Model, injecting driving representations learned from video generation into the planner to achieve stable and generalizable reasoning.
• Figure 2: Overview of the overall architecture of DriveLaW. The model first encodes historical observations (images, actions) into a unified latent world representation through a powerful video diffusion model. In order to improve the generation quality, we introduced the Noise Reinjection mechanism to explore and select the optimal generation path in the early stage of denoising. The denoised video latents produced by the Video DiT are then passed as conditioning signals to the action planner. Leveraging these latents, the lightweight Action DiT predicts future trajectories that are aligned with the visual scene evolution. In this chained design, the Video Model and Action Model share the same latent-space representation.
• Figure 3: Restoring Structural and Temporal Consistency via Noise Reinjection. This comparison highlights the impact of our method. The baseline generation shows significant degradation, including (a) blurring, (b) structural inconsistency, and (c) artifacts. By integrating noise reinjection, our model preserves sharp details, maintains object structures, and produces clean, artifact-free frames, demonstrating a crucial improvement in video quality.
• Figure 4: Qualitative Comparison with state-of-the-art driving world model. We compare DriveLaW with Epona zhang2025epona on nuScenes validation set. DriveLaW generates videos with (1) clearer vehicle details and more stable structural integrity, (2) well-preserved pedestrian shapes that remain easily identifiable, and (3) correct recognition and maintenance of inconspicuous objects (e.g., the yellow van), demonstrating superior visual quality, subject preservation, and semantic understanding.
• Figure 5: Qualitative analysis of latent representations. We visualize the quality of latent representations from three different feature sources: BEV features extracted from BEVFormer li2024bevformer's ResNet-101 backbone, VLM features from the pretrained Qwen2.5-VL model in ReCogDrive li2025recogdrive, and VGM (Video Generation Model) features from our DriveLaW-Video. To enable visual comparison, we apply PCA to reduce each representation to its top 3 principal components and map them to RGB channels. From top to bottom, each row displays: (1) the original input frame, (2) BEV features, (3) VLM features, and (4) VGM features, all upsampled to 1280×704 for visualization. While the BEV and VLM features appear diffuse, unstable, and exhibit irregular focus shifts, our VGM features are notably sharper, contain significantly less noise, and demonstrate superior semantic coherence with strong spatial structure awareness, even under severe driving motion.