Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WcDT: World-centric Diffusion Transformer for Traffic Scene Generation

Chen Yang, Yangfan He, Aaron Xuxiang Tian, Dong Chen, Jianhui Wang, Tianyu Shi, Arsalan Heydarian, Pei Liu

TL;DR

The paper tackles realistic, diverse multi-agent traffic scene generation for autonomous driving by introducing WcDT, a world-centric diffusion-transformer framework. It encodes historical agent behavior as Agent Move Statements, fuses scene context via a Transformer-based scene encoder, and decodes joint multimodal futures through a diffusion-based latent action module and a multimodal trajectory decoder. Key contributions include a world-centric representation, diffusion-transformer action encoding, temporal-spatial fusion in the scene encoder, and a robust loss combining diffusion, regression, and modality classification, achieving state-of-the-art realism and diversity on the Waymo dataset. The approach enables coherent, joint trajectory generation in a single inference and holds promise for more realistic ADS simulations and validation in complex urban traffic.

Abstract

In this paper, we introduce a novel approach for autonomous driving trajectory generation by harnessing the complementary strengths of diffusion probabilistic models (a.k.a., diffusion models) and transformers. Our proposed framework, termed the "World-Centric Diffusion Transformer"(WcDT), optimizes the entire trajectory generation process, from feature extraction to model inference. To enhance the scene diversity and stochasticity, the historical trajectory data is first preprocessed into "Agent Move Statement" and encoded into latent space using Denoising Diffusion Probabilistic Models (DDPM) enhanced with Diffusion with Transformer (DiT) blocks. Then, the latent features, historical trajectories, HD map features, and historical traffic signal information are fused with various transformer-based encoders that are used to enhance the interaction of agents with other elements in the traffic scene. The encoded traffic scenes are then decoded by a trajectory decoder to generate multimodal future trajectories. Comprehensive experimental results show that the proposed approach exhibits superior performance in generating both realistic and diverse trajectories, showing its potential for integration into automatic driving simulation systems. Our code is available at \url{https://github.com/yangchen1997/WcDT}.

WcDT: World-centric Diffusion Transformer for Traffic Scene Generation

TL;DR

The paper tackles realistic, diverse multi-agent traffic scene generation for autonomous driving by introducing WcDT, a world-centric diffusion-transformer framework. It encodes historical agent behavior as Agent Move Statements, fuses scene context via a Transformer-based scene encoder, and decodes joint multimodal futures through a diffusion-based latent action module and a multimodal trajectory decoder. Key contributions include a world-centric representation, diffusion-transformer action encoding, temporal-spatial fusion in the scene encoder, and a robust loss combining diffusion, regression, and modality classification, achieving state-of-the-art realism and diversity on the Waymo dataset. The approach enables coherent, joint trajectory generation in a single inference and holds promise for more realistic ADS simulations and validation in complex urban traffic.

Abstract

In this paper, we introduce a novel approach for autonomous driving trajectory generation by harnessing the complementary strengths of diffusion probabilistic models (a.k.a., diffusion models) and transformers. Our proposed framework, termed the "World-Centric Diffusion Transformer"(WcDT), optimizes the entire trajectory generation process, from feature extraction to model inference. To enhance the scene diversity and stochasticity, the historical trajectory data is first preprocessed into "Agent Move Statement" and encoded into latent space using Denoising Diffusion Probabilistic Models (DDPM) enhanced with Diffusion with Transformer (DiT) blocks. Then, the latent features, historical trajectories, HD map features, and historical traffic signal information are fused with various transformer-based encoders that are used to enhance the interaction of agents with other elements in the traffic scene. The encoded traffic scenes are then decoded by a trajectory decoder to generate multimodal future trajectories. Comprehensive experimental results show that the proposed approach exhibits superior performance in generating both realistic and diverse trajectories, showing its potential for integration into automatic driving simulation systems. Our code is available at \url{https://github.com/yangchen1997/WcDT}.
Paper Structure (19 sections, 12 equations, 5 figures, 5 tables)

This paper contains 19 sections, 12 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Motion prediction-based methods
  4. Generative model-based methods
  5. Methodology
  6. Traffic scene representation
  7. Action Diffusion
  8. Scene Encoder
  9. Trajectory Decoder
  10. Loss functions
  11. Experiments and Results
  12. Experimental Setup
  13. Comparison with state-of-the-art methods
  14. Ablation studies on diffusion model
  15. Ablation studies on traffic scene encoder
  16. ...and 4 more sections

Figures (5)

  • Figure 1: World-centric model and agent-centric model: (a) The conventional "Agent-centric model" is common in trajectory prediction, including on the Sim Agents leaderboard. (b) Our approach replaces complex coordinate transformations with position embeddings, enhancing efficiency in multi-scenario, multi-agent trajectory generation.
  • Figure 2: Overview of WcDT, which consists of the following modules: (a) Agent action generation and agent to agent cross attention blocks; (b) The traffic scene encoder extracts temporal and spatial features in the traffic scene, including: other agents, traffic signals, HD maps; (c) The multimodal trajectory decoder is used to generate possible future actions for all predicted agents.
  • Figure 3: Overview of the developed conditional DiT blocks and illustration of DiT block processing, where action latent features are integrated with map, object, and traffic light using multi-head attention for traffic scene generation.
  • Figure 4: Illustration of the trajectory generation process.
  • Figure 5: Visualization results of the ground truth and WTSGM-generated trajectories in cruise scenarios.