TransDiffuser: Diverse Trajectory Generation with Decorrelated Multi-modal Representation for End-to-end Autonomous Driving

TL;DR

This paper tackles mode collapse in diffusion-based trajectory generation for end-to-end autonomous driving by removing reliance on predefined trajectory vocabularies or scene priors. It proposes TransDiffuser, an encoder–decoder diffusion model that conditions on frozen, multi-modal scene and motion representations and introduces a lightweight multi-modal representation decorrelation loss to enrich the latent space and increase trajectory diversity. On NAVSIM, it achieves state-of-the-art PDMS (approximately 94.85) without anchors or priors, while qualitative results show more diverse, plausible trajectories that better explore drivable space. The work offers a practical, anchor-free approach with strong generalization potential for unseen scenarios and provides a framework for future integration with preference optimization and vision-language-action systems.

Abstract

In recent years, diffusion models have demonstrated remarkable potential across diverse domains, from vision generation to language modeling. Transferring its generative capabilities to modern end-to-end autonomous driving systems has also emerged as a promising direction. However, existing diffusion-based trajectory generative models often exhibit mode collapse where different random noises converge to similar trajectories after the denoising process.Therefore, state-of-the-art models often rely on anchored trajectories from pre-defined trajectory vocabulary or scene priors in the training set to mitigate collapse and enrich the diversity of generated trajectories, but such inductive bias are not available in real-world deployment, which can be challenged when generalizing to unseen scenarios. In this work, we investigate the possibility of effectively tackling the mode collapse challenge without the assumption of pre-defined trajectory vocabulary or pre-computed scene priors. Specifically, we propose TransDiffuser, an encoder-decoder based generative trajectory planning model, where the encoded scene information and motion states serve as the multi-modal conditional input of the denoising decoder. Different from existing approaches, we exploit a simple yet effective multi-modal representation decorrelation optimization mechanism during the denoising process to enrich the latent representation space which better guides the downstream generation. Without any predefined trajectory anchors or pre-computed scene priors, TransDiffuser achieves the PDMS of 94.85 on the closed-loop planning-oriented benchmark NAVSIM, surpassing previous state-of-the-art methods. Qualitative evaluation further showcases TransDiffuser generates more diverse and plausible trajectories which explore more drivable area.

Figures (5)

• Figure 1: Main approaches for end-to-end autonomous driving.
• Figure 2: Overview of the proposed TransDiffuser architecture. We freeze the parameters of the scene encoder.
• Figure 3: Illustrations on the multi-modal representation decorrelation process. (d) shows top 100 largest singular values of the correlation matrix. We observe the decorrelation mechanism helps to better utilize the representation space before action decoding (shown in Figure \ref{['fig:framework']}).
• Figure 4: Visualization of single-mode trajectories.
• Figure 5: Visualization on multi-mode diverse trajectories.