DriveGen: Towards Infinite Diverse Traffic Scenarios with Large Models

TL;DR

Many traffic simulators rely on limited real-world data, constraining the diversity of scenarios available for autonomous driving optimization. DriveGen presents a two-stage framework in which an initialization phase uses LLMs with retrieval to generate map and vehicle assets, and a rollout phase uses visual-language reasoning together with a diffusion planner to generate diverse, realistic trajectories. It also introduces DriveGen-CS, an automatic corner-case generation pipeline that leverages driving-failure prompts to generate challenging cases without retraining. Empirical results show that DriveGen's outputs achieve higher diversity and realism than state-of-the-art baselines, and downstream optimizations using DriveGen traffic improve the performance of typical driving algorithms.

Abstract

Microscopic traffic simulation has become an important tool for autonomous driving training and testing. Although recent data-driven approaches advance realistic behavior generation, their learning still relies primarily on a single real-world dataset, which limits their diversity and thereby hinders downstream algorithm optimization. In this paper, we propose DriveGen, a novel traffic simulation framework with large models for more diverse traffic generation that supports further customized designs. DriveGen consists of two internal stages: the initialization stage uses large language model and retrieval technique to generate map and vehicle assets; the rollout stage outputs trajectories with selected waypoint goals from visual language model and a specific designed diffusion planner. Through this two-staged process, DriveGen fully utilizes large models' high-level cognition and reasoning of driving behavior, obtaining greater diversity beyond datasets while maintaining high realism. To support effective downstream optimization, we additionally develop DriveGen-CS, an automatic corner case generation pipeline that uses failures of the driving algorithm as additional prompt knowledge for large models without the need for retraining or fine-tuning. Experiments show that our generated scenarios and corner cases have a superior performance compared to state-of-the-art baselines. Downstream experiments further verify that the synthesized traffic of DriveGen provides better optimization of the performance of typical driving algorithms, demonstrating the effectiveness of our framework.