Enhancing Autonomous Driving Safety with Collision Scenario Integration

TL;DR

This work tackles autonomous driving safety under hazardous, collision-prone scenarios by addressing collision-data scarcity and the shortcomings of imitation-learning-based training. It introduces CollisionGen, a scalable pipeline that generates realistic collision scenarios from natural language prompts and a map-conditional generative model, producing Collision2k, a diverse dataset of approximately 2,000 collision cases. Complementing this, SafeFusion trains neural planners with both regular and collision data using a fixed trajectory vocabulary and Multi-target Hydra-Distillation, removing imitation-learning reliance and improving safety metrics by substantial margins (e.g., up to 63.5% in safety and 56% overall). The approach yields strong improvements in collision avoidance while preserving performance on regular driving, providing a scalable, data-efficient path toward safer autonomous driving systems.

Abstract

Autonomous vehicle safety is crucial for the successful deployment of self-driving cars. However, most existing planning methods rely heavily on imitation learning, which limits their ability to leverage collision data effectively. Moreover, collecting collision or near-collision data is inherently challenging, as it involves risks and raises ethical and practical concerns. In this paper, we propose SafeFusion, a training framework to learn from collision data. Instead of over-relying on imitation learning, SafeFusion integrates safety-oriented metrics during training to enable collision avoidance learning. In addition, to address the scarcity of collision data, we propose CollisionGen, a scalable data generation pipeline to generate diverse, high-quality scenarios using natural language prompts, generative models, and rule-based filtering. Experimental results show that our approach improves planning performance in collision-prone scenarios by 56\% over previous state-of-the-art planners while maintaining effectiveness in regular driving situations. Our work provides a scalable and effective solution for advancing the safety of autonomous driving systems.

Figures (4)

• Figure 1: The red car represents the ego vehicle controlled by the planners, while the blue car represents the other vehicle. Previous planners struggle with dangerous scenarios due to the absence of collision data in real-world regular datasets. With CollisionGen and SafeFusion, planners achieve effective collision avoidance.
• Figure 2: The pipeline begins by taking text descriptions of collision scenarios as input. A generator with a language interpreter and a generative transformer is then applied, followed by the use of predefined rules and a PDM simulator dauner2023parting to filter out qualified collision scenarios. These filtered scenarios are subsequently used for the training and evaluation processes of planners.
• Figure 3: SafeFusion aims to improve the safety of neural planners by integrating collision data into training, using a planning vocabulary and Multi-Target Hydra-Distillation to address the challenge of unavailable collision-avoidance trajectories in collision data.
• Figure 4: Visualizations of planners.