WorldRFT: Latent World Model Planning with Reinforcement Fine-Tuning for Autonomous Driving

TL;DR

WorldRFT tackles the misalignment between reconstruction-focused latent world models and planning needs in autonomous driving by introducing a planning-oriented latent WM with a Spatial-aware World Encoder, Hierarchical Planning Refinement, and safety-oriented Reinforcement Learning Fine-Tuning. It leverages VGGT geometric priors, probabilistic target regions, and local-aware refinement, with GRPO-based optimization to enhance collision avoidance. The approach achieves state-of-the-art performance on nuScenes and NavSim, notably reducing collision rates and approaching LiDAR-based methods using only camera inputs. The results highlight the value of integrating geometric priors and planning-centric learning with explicit safety optimization for robust autonomous driving.

Abstract

Latent World Models enhance scene representation through temporal self-supervised learning, presenting a perception annotation-free paradigm for end-to-end autonomous driving. However, the reconstruction-oriented representation learning tangles perception with planning tasks, leading to suboptimal optimization for planning. To address this challenge, we propose WorldRFT, a planning-oriented latent world model framework that aligns scene representation learning with planning via a hierarchical planning decomposition and local-aware interactive refinement mechanism, augmented by reinforcement learning fine-tuning (RFT) to enhance safety-critical policy performance. Specifically, WorldRFT integrates a vision-geometry foundation model to improve 3D spatial awareness, employs hierarchical planning task decomposition to guide representation optimization, and utilizes local-aware iterative refinement to derive a planning-oriented driving policy. Furthermore, we introduce Group Relative Policy Optimization (GRPO), which applies trajectory Gaussianization and collision-aware rewards to fine-tune the driving policy, yielding systematic improvements in safety. WorldRFT achieves state-of-the-art (SOTA) performance on both open-loop nuScenes and closed-loop NavSim benchmarks. On nuScenes, it reduces collision rates by 83% (0.30% -> 0.05%). On NavSim, using camera-only sensors input, it attains competitive performance with the LiDAR-based SOTA method DiffusionDrive (87.8 vs. 88.1 PDMS).

Figures (8)

• Figure 1: Performance comparison of SOTA methods on open-loop nuScenes and closed-loop NavSim benchmarks.
• Figure 2: WorldRFT consists of three key modules: 1) Spatial-aware World Encoder (yellow) that extracts geometry-rich latent world representations from RGB images; 2) Hierarchical Planning Refinement Module (green) that efficiently captures critical information highly correlated with driving decisions through a refined planning task design; 3) Safety-aware Reinforcement Learning Fine-tuning phase (blue) that generates safer planning outcomes via reinforcement learning optimization. These modules work synergistically to deliver high-quality end-to-end autonomous driving planning.
• Figure 3: Architecture of the Local-aware Iterative Refinement module. The module refines preliminary planning results $(\mu,b), T_{\text{path}}, T_{\text{traj}}$ through $K$ iterations. Using $T_{\text{traj}}$ as an example, each iteration: (1) encodes global planning states, (2) projects trajectory points via camera parameters, (3) samples local features using deformable convolution, and (4) fuses them with global information and uncertainty representation. Residual connections enable incremental updates for adaptive local adjustment.
• Figure 4: Visualization of planning trajectories, where the red line is the ground truth, the blue line is the pretrained-only trajectory, and the green line shows the RFT-trajectory. Obviously, the pre-trained trajectory dangerously approaches obstacles, risking collisions. In contrast, the RFT-trajectory proactively adjusts to maintain safe distances, avoiding collisions.
• Figure 5: Visualization of the original image (top), along with attention maps for the baseline (middle) and WorldRFT (bottom), where blue represents weak attention and red indicates strong attention. The maps reveal that WorldRFT focuses on surrounding planning-critical agents such as vehicles, which are largely overlooked by the baseline method.