AdaWM: Adaptive World Model based Planning for Autonomous Driving

TL;DR

AdaWM addresses finetuning degradation in world-model–based autonomous driving by jointly identifying whether dynamics or policy mismatches dominate under distribution shift and applying alignment-driven, efficient updates accordingly. It introduces LoRA-based low-rank updates for the dynamics model and a convex sub-unit policy scheme to enable selective adaptation, guided by mismatch metrics derived from TV distances. The approach yields substantial improvements in planning robustness and safety metrics (e.g., TTC and SR) across diverse CARLA tasks compared to strong baselines. These results highlight the practical value of adaptive finetuning strategies for real-world driving systems and pave the way for broader application of mismatch-aware adaptation.

Abstract

World model based reinforcement learning (RL) has emerged as a promising approach for autonomous driving, which learns a latent dynamics model and uses it to train a planning policy. To speed up the learning process, the pretrain-finetune paradigm is often used, where online RL is initialized by a pretrained model and a policy learned offline. However, naively performing such initialization in RL may result in dramatic performance degradation during the online interactions in the new task. To tackle this challenge, we first analyze the performance degradation and identify two primary root causes therein: the mismatch of the planning policy and the mismatch of the dynamics model, due to distribution shift. We further analyze the effects of these factors on performance degradation during finetuning, and our findings reveal that the choice of finetuning strategies plays a pivotal role in mitigating these effects. We then introduce AdaWM, an Adaptive World Model based planning method, featuring two key steps: (a) mismatch identification, which quantifies the mismatches and informs the finetuning strategy, and (b) alignment-driven finetuning, which selectively updates either the policy or the model as needed using efficient low-rank updates. Extensive experiments on the challenging CARLA driving tasks demonstrate that AdaWM significantly improves the finetuning process, resulting in more robust and efficient performance in autonomous driving systems.

Figures (10)

• Figure 1: Performance comparison of different finetuning strategies in the left turn with moderate traffic flow task.
• Figure 2: A sketch of adaptive world model based planning (AdaWM): During pretraining, a dynamics model and a planning policy are learned offline. For online adaptation, at each finetuning step $t$, AdaWM first identifies the more dominating mismatch that causes the performance degradation and then carries out alignment-driven finetuning accordingly.
• Figure 3: Learning curves of different finetuning strategies in four evaluation tasks.
• Figure 4: The mismatches of the dynamics model and policy during the finetuning.
• Figure 5: The mismatches of the dynamics model and policy with different value of $C$.

Theorems & Definitions (4)

• Theorem 1
• Lemma 1
• Lemma 2: Generalization Error of RNN
• Lemma 3