Controllable risk scenario generation from human crash data for autonomous vehicle testing

TL;DR

The paper addresses the challenge of testing autonomous vehicles under rare safety-critical conditions by unifying nominal driving behavior with risk-prone behavior. It introduces CRAG, a framework that learns a structured latent risk space from crash data via a variational autoencoder and uses optimization in latent space to generate controllable risk scenarios within closed-loop simulations. By fusing a risk-aware latent representation with optimization-guided state transitions, CRAG achieves diverse, realistic, and targeted safety-critical scenarios while maintaining fidelity to real-world crash patterns. Experimental results demonstrate improved diversity and controllability in both one-way and intersection scenarios, enabling scalable, targeted evaluation of AV robustness.

Abstract

Ensuring the safety of autonomous vehicles (AV) requires rigorous testing under both everyday driving and rare, safety-critical conditions. A key challenge lies in simulating environment agents, including background vehicles (BVs) and vulnerable road users (VRUs), that behave realistically in nominal traffic while also exhibiting risk-prone behaviors consistent with real-world accidents. We introduce Controllable Risk Agent Generation (CRAG), a framework designed to unify the modeling of dominant nominal behaviors and rare safety-critical behaviors. CRAG constructs a structured latent space that disentangles normal and risk-related behaviors, enabling efficient use of limited crash data. By combining risk-aware latent representations with optimization-based mode-transition mechanisms, the framework allows agents to shift smoothly and plausibly from safe to risk states over extended horizons, while maintaining high fidelity in both regimes. Extensive experiments show that CRAG improves diversity compared to existing baselines, while also enabling controllable generation of risk scenarios for targeted and efficient evaluation of AV robustness.

Figures (15)

• Figure 1: Traditional AV testing isolates normal and risky driving data, hindering the generation of safety-critical scenarios in simulation. CRAG leverages both data types to enhance the realism and quality of generated risky scenarios.
• Figure 2: Proposed risky testing scenario generation pipeline.
• Figure 3: Agent poses represented in the AV coordinate system: AV (blue) and BV (green).
• Figure 4: Illustration of the risk scenario generation framework. The system integrates a motion predictor, VAE-based latent representation, and latent-space optimization to produce dynamically plausible and controllable risky states.
• Figure 5: Generation of a risky scenario using the baseline algorithm, where the mode with the highest risk level is selected.