Direct Preference Optimization-Enhanced Multi-Guided Diffusion Model for Traffic Scenario Generation

TL;DR

The paper tackles realistic, diverse, and controllable traffic scenario generation for autonomous vehicle testing by proposing MuDi-Pro, a diffusion-transformer backbone augmented with a guidance conditional layer and Direct Preference Optimization (DPO) for fine-tuning. It learns a data-driven traffic prior and supports multiple guides via a single model, using a TRACE-inspired guidance mechanism and a DPO loss to optimize preferences over sample pairs $(T_w^0, T_l^0)$ drawn from dataset $\mathcal{D}$, wrapped in $L_{DPO}$. Evaluated on the nuScenes dataset, MuDi-Pro demonstrates improved realism, diversity, and controllability compared to STRIVE and CTG baselines, while effectively leveraging guided sampling without heavy gradient computations. The approach offers practical impact for testing autonomous vehicles by enabling realistic, diverse, and rule-compliant traffic scenario generation with configurable guidance and efficient fine-tuning.

Abstract

Diffusion-based models are recognized for their effectiveness in using real-world driving data to generate realistic and diverse traffic scenarios. These models employ guided sampling to incorporate specific traffic preferences and enhance scenario realism. However, guiding the sampling process to conform to traffic rules and preferences can result in deviations from real-world traffic priors and potentially leading to unrealistic behaviors. To address this challenge, we introduce a multi-guided diffusion model that utilizes a novel training strategy to closely adhere to traffic priors, even when employing various combinations of guides. This model adopts a multi-task learning framework, enabling a single diffusion model to process various guide inputs. For increased guided sampling precision, our model is fine-tuned using the Direct Preference Optimization (DPO) algorithm. This algorithm optimizes preferences based on guide scores, effectively navigating the complexities and challenges associated with the expensive and often non-differentiable gradient calculations during the guided sampling fine-tuning process. Evaluated using the nuScenes dataset our model provides a strong baseline for balancing realism, diversity and controllability in the traffic scenario generation.

Figures (6)

• Figure 1: Overview of MuDi-Pro Training Process. MuDi-Pro employs a two-step training process.
• Figure 2: Model architecture of training and inference. (a) The model is trained to predict clean trajectories from noisy ones. The input trajectory and traffic information are first encoded, then processed through Transformer blocks. (b) In the inference phase, classifier-free sampling controls the amount of future feature information provided to the sample, while guided sampling directs the sample generation towards the desired outcome during the denoising process.
• Figure 3: Fine-tuning Framework. Fine-tuning is conducted in two phases: (a) The guidance conditional layer fine-tunes a segment of the transformer based on chosen guidance. (b) DPO further fine-tunes the target model using DPO loss.
• Figure 4: Examples of sample data exhibiting collisions from the onset of traffic, with collision points highlighted by red circles. Most of the vehicles with collisions are parked and have stopped slightly off roads.
• Figure 5: Qualitative results of MuDi and MuDi-Pro. (a) Qualitative results demonstrate that our model can produce diverse samples with varying w. (b) Qualitative results of comparing MuDi-Pro with MuDi. MuDi-Pro produces plausible and realistic trajectories in scenes where MuDi fails to do so. (c) Qualitative diversity results of MuDi-Pro. The generated trajectories, from both the sample mode and the reconstruct mode, tends to exhibit multi-modal characteristics. Significant differences are highlighted with red squares.